Using summaries of '011235813/hierarchical-marl' files from Context, write repository README. Focus on the functionalities and features. There is no need to describe the dependencies and setup. The README should provide answers to the following questions:

* What machine learning problem does this repository tackle?
The repository tackles the problem of training a hierarchical multi-agent reinforcement learning (HSD) algorithm, which involves training both a high-level policy that selects subgoals and a low-level policy that takes actions based on those subgoals. The file also includes code for computing intrinsic rewards and updating the high-level state.
* What kind of data does it use?
The repository uses a dataset of observations collected during training episodes to train both the high-level policy and low-level policies using stochastic gradient descent (SGD).


File `alg/train_hsd.py`:

* Implements the training loop for the hierarchical multi-agent reinforcement learning (HSD) algorithm, which involves training both a high-level policy that selects subgoals and a low-level policy that takes actions based on those subgoals. The file also includes code for computing intrinsic rewards and updating the high-level state.
* Uses the `hierarchical_marl` library to interact with the environment and compute intrinsic rewards.
* Trains both the high-level policy and low-level policies using stochastic gradient descent (SGD) on a dataset of observations collected during training episodes.

File `alg/hierarchical_marl.py`:

* Implements the hierarchical multi-agent reinforcement learning (HSD) algorithm, which involves training both a high-level policy that selects subgoals and a low-level policy that takes actions based on those subgoals. The file also includes code for computing intrinsic rewards and updating the high-level state.
* Uses the `hierarchical_marl` library to interact with the environment and

q learning
multi agent reinforcement learning
reinforcement learning


alg/alg_hsd.py, alg/alg_hsd_scripted.py, alg/alg_iql.py, alg/train_hsd.py, alg/train_hsd_scripted.py, Qmix_single, Q_low, Q_high, Qmix_mixer, test_hierarchy, train_policy_high, train_policy_low, train_step_low, dim_role_space, arena_min_x


alg/alg_hsd.py, alg/alg_hsd_scripted.py, alg/alg_iql.py, alg/train_hsd.py, alg/train_hsd_scripted.py, Qmix_single, Q_low, Q_high, Qmix_mixer, test_hierarchy, train_policy_high, train_policy_low, train_step_low, dim_role_space, arena_min_x## tasks:
#hsd, iql, roboschool,


hsd, iql, roboschool,


alg/alg_hsd.py:
⋮...
│class Alg(object):
│
│    def __init__(self, config_alg, config_h, n_agents, l_state, l_obs, l_action, l_z, nn):
│        """
│        Args:
│            config_alg: dictionary of general RL params
│            config_h: dictionary of HSD params
│            n_agents: number of agents on the team controlled by this alg
│            l_state, l_obs, l_action, l_z: int
│            nn: dictionary with neural net sizes
│        """
│        self.l_state = l_state
⋮...
│    def process_actions(self, n_steps, actions, n_actions):
⋮...
│    def process_batch(self, batch):
⋮...

alg/alg_hsd_scripted.py:
⋮...
│class Alg(object):
│
│    def __init__(self, alg_name, config_alg, n_agents, l_state, l_obs, l_action, N_roles, nn):
│        """
│        Args:
│            alg_name: currently supports 'hsd-scripted' for QMIX at high level or 'mara-c' for Q-le
│            config_alg: dictionary of general RL params
│            n_agents: number of agents on the team controlled by this alg
│            l_state, l_obs, l_action, N_roles: int
│            nn: dictionary with neural net sizes
│        """
│        self.alg_name = alg_name
│
⋮...
│    def process_actions(self, n_steps, actions, n_actions):
⋮...
│    def process_batch(self, batch):
⋮...

alg/alg_iql.py:
⋮...
│class Alg(object):
│
│    def __init__(self, config_alg, n_agents, l_state, l_obs, l_action, nn):
│        """
│        Args:
│            config_alg: dictionary of general RL params
│            n_agents: number of agents on the team controlled by this alg
│            l_state, l_obs, l_action: int
│            nn: dictionary with neural net sizes
│        """
│        self.l_state = l_state
│        self.l_obs = l_obs
⋮...
│    def process_actions(self, n_steps, actions):
⋮...
│    def process_batch(self, batch):
⋮...

alg/alg_qmix.py:
⋮...
│class Alg(object):
│
│    def __init__(self, config_alg, n_agents, l_state, l_obs, l_action, nn):
│        """
│        Args:
│            config_alg: dictionary of general RL params
│            n_agents: number of agents on the team controlled by this alg
│            l_state, l_obs, l_action: int
│            nn: dictionary with neural net sizes
│        """
│        self.l_state = l_state
│        self.l_obs = l_obs
⋮...
│    def create_networks(self):
│
⋮...
│    def create_summary(self):
│
⋮...
│    def process_actions(self, n_steps, actions):
⋮...
│    def process_batch(self, batch):
⋮...

alg/networks.py:
⋮...
│def get_variable(name, shape):
│
⋮...
│def Qmix_single(obs, n_h1, n_h2, n_actions):
⋮...
│def Qmix_mixer(agent_qs, state, state_dim, n_agents, n_h_mixer):
⋮...

alg/replay_buffer.py:
⋮...
│class Replay_Buffer():
│
│    def __init__(self, size=1e6):
│        self.memory = []
│        self.maxsize = int(size)
⋮...
│    def sample(self, length):
⋮...

env/env_wrapper.py:
⋮...
│class Env(object):
│
│    def __init__(self, config_env, config_main, test=False, N_roles=None):
│        """Set up variables
│
│        Args:
│            config_env: dictionary of environment parameters
│            config_main: dictionary of main experiment parameters
│            test: if True, turns on measurements (e.g. for testing a policy)
│            N_roles: only used for extra measurements during test
│        """
│        self.N_home = config_env['num_home_players']
⋮...
│    def normalize(self, x, z):
⋮...
│    def rel_norm_pos(self, x, z, team):
⋮...
│    def process_actions(self, actions_int, roles=None, team='home'):
⋮...
│    def step(self, actions_int, roles=None, measure=False):
⋮...
│    def reset(self):
⋮...


lio/alg/inequity_aversion.py:
⋮...
│class InequityAversion(object):
│
│    def __init__(self, alpha, beta, gamma, e, n_agents):
│        """Initialization.
│
│        Args:
│            alpha: aversion to disadvantageous inequity
│            beta: aversion to advantageous inequity
│            gamma: discount factor
│            e: lambda in the paper
│            n_agents: number of agents
│        """
⋮...
│    def reset(self):
⋮...

lio/alg/networks.py:
⋮...
│def conv(t_input, scope, n_filters=6, k=(3, 3), s=(1, 1), data_format='NHWC'):
│
⋮...
│def convnet(t_input, f=6, k=(3, 3), s=(1, 1)):
│
⋮...

lio/alg/train_pg.py:
⋮...
│class Buffer(object):
│
│    def __init__(self, n_agents):
│        self.n_agents = n_agents
⋮...
│    def reset(self):
⋮...

lio/alg/train_ssd.py:
⋮...
│class Buffer(object):
│
⋮...
│    def __init__(self, n_agents):
│
│        self.n_agents = n_agents
│
⋮...
│    def reset(self):
│
⋮...

lio/env/room_agent.py:
⋮...
│class Actor(object):
│
⋮...
│    def __init__(self, agent_id, n_agents, l_obs):
│
│
│
│        self.agent_id = agent_id
│
│        self.l_obs = l_obs
│
│        self.n_agents = n_agents
│
⋮...
│    def get_obs(self, state, observe_given=True):
│
⋮...

lio/env/room_asymmetric_lola.py:
⋮...
│class EscapeRoomAsym(gym.Env):
│    """
│    A two-agent vectorized environment for asymmetric Escape Room game.
│    Possible actions for A1 are move to Start, Door
│    Possible actions for A2 are move to Start, Lever
⋮...
│    def get_door_status(self, actions):
⋮...
│    def calc_reward(self, actions, door_open):
⋮...
│    def get_obs(self):
⋮...

lio/env/room_symmetric.py:
⋮...
│class Env(object):
│
⋮...
│    def __init__(self, config_env):
│
│
│
│        self.config = config_env
│
│
│
│        self.n_agents = self.config.n_agents
│
⋮...
│    def get_door_status(self, actions):
│
⋮...
│    def calc_reward(self, actions, given_rewards, door_open):
│
⋮...
│    def get_obs(self):
│
⋮...

lio/env/room_symmetric_baseline.py:
⋮...
│class EscapeRoom(gym.Env):
│    """
│    A two-agent vectorized environment for symmetric Escape Room game.
│    Possible actions for each agent are move to Key, Start, Door
⋮...
│    def get_door_status(self, actions):
⋮...
│    def calc_reward(self, actions, door_open):
⋮...
│    def get_obs(self):
⋮...
│    def reset(self):
⋮...

lio/env/room_symmetric_centralized.py:
⋮...
│class EscapeRoom(object):
│
│    def __init__(self, config):
│
│        self.name = 'er'
│        # Internal value for convenience
│        self._n_agents = config.n_agents
│        # Only 2 and 3-player are supported
│        assert self._n_agents == 2 or self._n_agents == 3
│        self.max_steps = config.max_steps
│        # Only support (N=2,M=1) and (N=3,M=2)
│        self.min_at_lever = config.min_at_lever
⋮...
│    def get_door_status(self, actions):
⋮...
│    def calc_reward(self, actions, door_open):
⋮...
│    def get_obs(self):
⋮...
│    def reset(self):
⋮...

lio/utils/configdict.py:
⋮...
│class ConfigDict(dict):
⋮...

lio/utils/util.py:
⋮...
│def process_actions(actions, l_action):
│
⋮...
│def get_action_others_1hot(action_all, agent_id, l_action):
│
⋮...
│def get_action_others_1hot_batch(list_action_all, agent_id, l_action):
│
⋮...


learning, robotics, physics simulation,


# config.env.allow_giving = False
...
# if not None, a fixed global reference frame is used for all agents
# config.env.global_ref_point = [4, 4]  # cleanup_10x10
# config.env.global_ref_point = [3, 3]  # for cleanup_small
...
# ---------- For 10x10 map ----------
# config.env.obs_height = 15
# config.env.obs_width = 15
# -----------------------------------
# ---------- for 7x7 map ------------
...
# -----------------------------------
...
# config.env.view_size = 5  # for 10x10 map with global ref point
...
# config.env.view_size = 7  # 0.5(height - 1)
...
# 10x10 0.3 | small 0.5
# 10x10 0.4 | small 0.6
# 10x10 0.0 | small 0
# 10x10 0.5 | small 0.5

# Map from name of map to the largest column position
...
# where a cleaning beam fired from that position can clear waste
...
# entry (i,j) is reward that agent i gives to agent j
...
# add rewards received from others
...
# subtract rewards given to others
...
# When using discrete reward-giving actions,
...
# env_rewards may not account for the full cost, since
...
# we may have cost = reward_coeff * reward_value
...
# (see room_symmetric_baseline.py)
...
# The original env reward is recorded in info
...
# rewards_total += env_rewards
...
# These are the positions seen by the incentive function
...
# (i, j) is reward from i to j
...
# add rewards received from others
...
# subtract amount given to others
...
# assumes N=2
...
# Don't need to modify actual rewards since only the
...
# traces are needed for agents' observations
...
# Note that when using discrete reward-giving actions,
...
# env_rewards does not account for the full cost, since
...
# we may have cost = reward_coeff * reward_value (see ssd_discrete_reward.py)
...
# so the actual extrinsic reward is recorded in info
...
# assuming N=2
...
# given, assuming N=2
...
# Run scripted agent
...
# given to the scripted agent
...
# count of cooperation

class ActorCritic(object):
...
# -------------------
# Used only when agent's observation has both image and 1D vector parts
...
# -------------------
...

    def create_networks(self):
...
# target <- main
...
# target <- tau * main + (1-tau) * target
...

    def run_actor(self, obs, sess, epsilon, obs_v=None):
...

    def create_critic_train_op(self):
...

    def create_policy_gradient_op(self):
...

    def train(self, sess, buf, epsilon):
...
# Update value network

class ActorCritic(actor_critic.ActorCritic):
...

    def create_networks(self):
...
# Will be squashed by sigmoid later
...
# target <- main
...
# target <- tau * main + (1-tau) * target
...

    def give_reward(self, obs, action_all, sess):
...
# sigmoid
...

    def create_policy_gradient_op(self):
...
# movement part
...
# reward part
...
# Account for the change of variables due to passing through sigmoid and scaling
# The formula implemented here is p(y) = p(x) |det dx/dy | = p(x) |det 1/(dy/dx)|
...
# Assume pi(a,r|s) = pi(a|s)*pi(r|s)
...

    def train(self, sess, buf, epsilon):
...
# Update value network

# config.env.allow_giving = False
...
# if not None, a fixed global reference frame is used for all agents
# config.env.global_ref_point = [7, 7]  # for cleanup_wall_15x15
# config.env.global_ref_point = [3, 7]  # for cleanup_check
# for cleanup_small
# config.env.global_ref_point = [2, 2]  # for cleanup_tiny
# config.env.global_ref_point = None
...
# ---------- For 15x15 map ----------
# config.env.obs_height = 25
# config.env.obs_width = 25
# ---------- For 10x10 map ----------
# config.env.obs_height = 15
# config.env.obs_width = 15
# -----------------------------------
# ---------- for 7x7 map ------------
...
# -----------------------------------
# ---------- For 4x5 map ------------
# config.env.obs_height = 5
# config.env.obs_width = 5
# -----------------------------------
...
# config.env.view_size = 2
...
# config.env.view_size = 7  # 0.5(height - 1)
# config.env.view_size = 12
...
# orig 0.05 | 15x15 0.2 | 10x10 0.3 | small 0.5 | min 1.0
# orig 0.4 | 15x15 0.4 | 10x10 0.4 | small 0.6 | min 0.02
# orig 0 | 15x15 0 | 10x10 0.0 | small 0 | min 0.01
# orig 0.5 | 15x15 0.5 | 10x10 0.5 | small 0.5 | min 0.5
...
# what should this be?

# scale up sigmoid output

# default 3
...
# if not None, a fixed global reference frame is used for all agents
# config.env.global_ref_point = [4, 4]  # cleanup_10x10
# config.env.global_ref_point = [3, 3]  # for cleanup_small
...
# 'cleanup_small_sym'|'cleanup_10x10_sym'
# small: 50 | 10x10: 50
...
# If T, reward function takes in 1-hot representation of
# whether the other agent used the cleaning beam
# Else, observe the full 1-hot action of other agent
...
# ---------- For 10x10 map cleanup_10x10_sym ----------
# config.env.obs_height = 15
# config.env.obs_width = 15
# -----------------------------------
# ---------- For 7x7 map cleanup_small_sym ------------
...
# -----------------------------------
# scale up sigmoid output
...
# 0.5(height - 1)
...
# config.env.view_size = 7
...
# 10x10 0.3 | small 0.5
# 10x10 0.4 | small 0.6
# 10x10 0.0 | small 0.0
# 10x10 0.5 | small 0.5
...
# Keep this as False, before figuring out potential gradient issue with tf.minimum
...
# small 1e2, 10x10 1e3
...
# only used if asymmetric=True

class InequityAversion(object):
...

    def __init__(self, alpha, beta, gamma, e, n_agents):
...

    def reset(self):
...
# Temporally-smoothed rewards
...

    def compute_rewards(self, rewards_env):
...
# Update temporal smoothed rewards
...
# Iterate over agents and compute its inequity aversion reward
...
# Disadvantageous inequity


lio/alg/actor_critic.py, lio/alg/actor_critic_discrete_continuous.py, lio/alg/config_ipd_lio.py, lio/alg/config_room_lio.py, lio/alg/config_room_pg.py, actor_mlp, actor_image_vec, reward_mlp, vnet_mlp, vnet_image_vec, decentralized, train_opp_model, create_opp_modeling_op, add_action_all, add_r_given


* `config_room_pg.py`: This file contains configuration parameters for the Room environment used in the LIO algorithm. It specifies the size of the environment, the number of agents, and other parameters that affect the behavior of the agents.
* `config_ssd_lio.py`: This file contains configuration parameters for the SSD environment used in the LIO algorithm. It specifies the size of the environment, the number of agents, and other parameters that affect the behavior of the agents.
* `inequity_aversion.py`: This file implements an Inequity Aversion (IA) reward function, which is a modification of the standard LIO reward function that encourages agents to avoid collisions with each other. The IA reward function computes the inequity between two agents and assigns a negative reward if they are close enough to collide.
* `lio_model.py`: This file contains the implementation of the LIO model, which is a reinforcement learning algorithm that learns to control multiple autonomous robots in a shared workspace. The model uses a neural network to predict the optimal control inputs for each agent based on its current state and the states of other agents


This repository tackles the problem of multi-agent reinforcement learning for controlling multiple autonomous robots in a shared workspace. The LIO (Learning Inverse Optimization) algorithm is used to learn the optimal control inputs for each agent based on its current state and the states of other agents.

The repository uses data from the Room environment, which consists of a grid-based workspace with obstacles and goals for the agents to navigate. The number of agents and the size of the environment are specified in the configuration files `config_room_pg.py` and `config_ssd_lio.py`, respectively.

The repository also includes an Inequity Aversion (IA) reward function, which is a modification of the standard LIO reward function that encourages agents to avoid collisions with each other. The IA reward function computes the inequity between two agents and assigns a negative reward if they are close enough to collide.

Overall, this repository provides an implementation of the LIO model for controlling multiple autonomous robots in a shared workspace, using data from the Room environment and an Inequity Aversion reward function.


This repository tackles the problem of multi-agent reinforcement learning for controlling multiple autonomous robots in a shared workspace. The LIO (Learning Inverse Optimization) algorithm is used to learn the optimal control inputs for each agent based on its current state and the states of other agents. The repository uses data from the Room environment, which consists of a grid-based workspace with obstacles and goals for the agents to navigate. The number of agents and the size of the environment are specified in the configuration files `config_room_pg.py` and `config_ssd_lio.py`, respectively. The repository also includes an Inequity Aversion (IA) reward function, which is a modification of the standard LIO reward function that encourages agents to avoid collisions with each other. The IA reward function computes the inequity between two agents and assigns a negative reward if they are close enough to collide. Overall, this repository provides an implementation of the LIO model for controlling multiple autonomous robots in a shared workspace, using data from the Room environment and an Inequity Aversion reward function.


# Learning to Incentivize Others

This is the code for experiments in the paper [Learning to Incentivize Other Learning Agents](https://arxiv.org/abs/2006.06051). Baselines are included.

## Setup

- Python 3.6
- Tensorflow >= 1.12
- OpenAI Gym == 0.10.9
- Clone and `pip install` [Sequential Social Dilemma](https://github.com/011235813/sequential_social_dilemma_games), which is a fork from the [original](https://github.com/eugenevinitsky/sequential_social_dilemma_games) open-source implementation.
- Clone and `pip install` [LOLA](https://github.com/alshedivat/lola) if you wish to run this baseline.
- Clone this repository and run `$ pip install -e .` from the root.

## Navigation

* `alg/` - Implementation of LIO and PG/AC baselines
* `env/` - Implementation of the Escape Room game and wrappers around the SSD environment.
* `results/` - Results of training will be stored in subfolders here. Each independent training run will create a subfolder that contains the final Tensorflow model, and reward log files. For example, 5 parallel independent training runs would create `results/cleanup/10x10_lio_0`,...,`results/cleanup/10x10_lio_4` (depending on configurable strings in config files).
* `utils/` - Utility methods

## Examples

### Train LIO on Escape Room

* Set config values in `alg/config_room_lio.py`
* `cd` into the `alg` folder
* Execute training script `$ python train_multiprocess.py lio er`. Default settings conduct 5 parallel runs with different seeds.
* For a single run, execute `$ python train_lio.py er`.

### Train LIO on Cleanup

* Set config values in `alg/config_ssd_lio.py`
* `cd` into the `alg` folder
* Execute training script `$ python train_multiprocess.py lio ssd`.
* For a single run, execute `$ python train_ssd.py`.

## Citation

<pre>
@article{yang2020learning,
  title={Learning to incentivize other learning agents},
  author={Yang, Jiachen and Li, Ang and Farajtabar, Mehrdad and Sunehag, Peter and Hughes, Edward and Zha, Hongyuan},
  journal={Advances in Neural Information Processing Systems},
  volume={33},
  pages={15208--15219},
  year={2020}
}
</pre>

## License

See [LICENSE](LICENSE).

SPDX-License-Identifier: MIT


lio/alg/actor_critic.py, lio/alg/actor_critic_discrete_continuous.py, lio/alg/config_ipd_lio.py, lio/alg/config_room_lio.py, lio/alg/config_room_pg.py, actor_mlp, actor_image_vec, reward_mlp, vnet_mlp, vnet_image_vec, decentralized, train_opp_model, create_opp_modeling_op, add_action_all, add_r_given## tasks:
#learning, robotics, physics simulation,


base/IterativeRecommender.py:
⋮...
│class IterativeRecommender(Recommender):
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
⋮...
│    def readConfiguration(self):
⋮...

data/record.py:
⋮...
│class Record(object):
│    'data access control'
⋮...
│    def getId(self,obj,t):
⋮...
│    def contains(self, obj, t):
⋮...

recommender/advanced/CUNE.py:
⋮...
│class CUNE(IterativeRecommender):
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
⋮...
│    def readConfiguration(self):
⋮...

recommender/advanced/DMF.py:
⋮...
│class DMF(DeepRecommender):
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
⋮...
│    def readConfiguration(self):
⋮...

recommender/advanced/NeuTrans.py:
⋮...
│class NeuTrans(DeepRecommender):
│
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
│
⋮...
│    def readConfiguration(self):
│
⋮...

recommender/advanced/Song2vec.py:
⋮...
│class Song2vec(IterativeRecommender):
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
⋮...
│    def readConfiguration(self):
⋮...

recommender/cf/IPF.py:
⋮...
│class IPF(Recommender):
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
⋮...
│    def readConfiguration(self):
⋮...

recommender/cf/UserKNN.py:
⋮...
│class UserKNN(Recommender):
│    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
│        super(UserKNN, self).__init__(conf,trainingSet,testSet,fold)
│        self.userSim = SymmetricMatrix(len(self.data.name2id['user']))
⋮...
│    def readConfiguration(self):
⋮...

structure/symmetricMatrix.py:
⋮...
│class SymmetricMatrix(object):
│    def __init__(self, shape):
│        self.symMatrix = {}
⋮...
│    def set(self,i,j,val):
⋮...
│    def contains(self,i,j):
⋮...

tool/TSNE.py:
⋮...
│def Hbeta(D=np.array([]), beta=1.0):
⋮...

tool/config.py:
⋮...
│class Config(object):
│    def __init__(self,fileName):
│        self.config = {}
⋮...
│    def contains(self,key):
⋮...
│class LineConfig(object):
│    def __init__(self,content):
│        self.line = content.strip().split(' ')
│        self.options = {}
│        self.mainOption = False
│        if self.line[0] == 'on':
│            self.mainOption = True
│        elif self.line[0] == 'off':
│            self.mainOption = False
│        for i,item in enumerate(self.line):
│            if (item.startswith('-') or item.startswith('--')) and  not item[1:].isdigit():
⋮...
│    def contains(self,key):
⋮...

tool/file.py:
⋮...
│class FileIO(object):
│    def __init__(self):
⋮...
│    @staticmethod
│    def writeFile(dir,file,content,op = 'w'):
⋮...

tool/qmath.py:
⋮...
│def common(x1,x2):
⋮...
│def cosine_sp(x1,x2):
⋮...
│def pearson_sp(x1,x2):
⋮...
│def euclidean(x1,x2):
⋮...

yue.py:
⋮...
│def run(measure,algor,order):
⋮...


collaborative filtering, general recommendation, general reinforcement learning,


1. `CUNE.py`: This file contains a class named `CUNE` that inherits from the `IterativeRecommender` class. It implements a collaborative filtering model using the C-U-NET algorithm, which is an extension of the User-based and Item-based collaborative filtering models. The class reads in configuration files to set up the model and training parameters.
2. `advanced/CUNE.py`: This file contains a similar implementation of the `CUNE` class as `CUNE.py`, but it also includes additional functionality for generating top-k friends for each user and calculating global user preferences. It also prepares sets of positive, negative, and neutral items for each user.
3. `data/DataReader.py`: This file contains a class named `DataReader` that reads in data from a CSV file and preprocesses it to create a sparse matrix representation of the data. The class can also split the data into training and testing sets based on a specified fold number.


yue.py, base/IterativeRecommender.py, data/record.py, evaluation/measure.py, recommender/advanced/ABLAH.py, pearson_sp, cosine_sp, Yue, ABLAH, CUNE, x2p, common, Hbeta, eig, real


Using summaries of '0411tony/Yue' files from Context, write repository README. Focus on the functionalities and features. There is no need to describe the dependencies and setup. The README should provide answers to the following questions:

* What machine learning problem does this repository tackle?
	+ This repository tackles the problem of collaborative filtering using the C-U-NET algorithm, which is an extension of the User-based and Item-based collaborative filtering models.
* What kind of data does it use?
	+ The repository uses a sparse matrix representation of data from a CSV file.


class CoFactor(IterativeRecommender):
...

    def __init__(self, conf, trainingSet=None, testSet=None, fold='[1]'):
...

    def readConfiguration(self):
...
#the number of negative samples
...

    def printAlgorConfig(self):
...

    def initModel(self):
...
#constructing SPPMI matrix
...
#for larger data set has many items, the process will be time consuming
...
# maxx = -1
...
#normalize
...

    def buildModel(self):
...
#Theta
#Beta
# bias value of item
# bias value of context
# context embedding
...
# sparse matrix
...
# self.loss += (self.X * self.X).sum() + (self.Y * self.Y).sum()
...
# if self.isConverged(iteration):
...

    def predictForRanking(self,u):

#coding:utf8
...
class APR(DeepRecommender):
...

    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
...
# APR：Adversarial Personalized Ranking for Recommendation
...

    def readConfiguration(self):
...

    def _create_variables(self):
...
#perturbation vectors
...
#parameters
...

    def _create_inference(self):
...

    def _create_adv_inference(self):
...

    def _create_adversarial(self):
...
#get gradients of Delta
...
# convert the IndexedSlice Data to Dense Tensor
...
# normalization: new_grad = (grad / |grad|) * eps
...

    def _create_loss(self):
...
#loss of adversarial training
...

    def _create_optimizer(self):
...

    def initModel(self):
...

    def next_batch(self):
...
#negative sampling
...

    def buildModel(self):
...
# train the model until converged
...
# start adversarial training

class Record(object):
...

    def __init__(self,config,trainingSet,testSet):
...
#key:artist id, value:{album id1:1, album id2:1 ...}
#
#
#
#
#user data in training set. form: {user:[record1,record2]}
# track data in training set. form: {track:[record1, record2]}
#user data in test set. form: {user:{recommenedObject1:1,recommendedObject:1}}
...
#used to store the mean values of users's listen tims
...

    def globalAverage(self):
...

    def computeUserMean(self):
...

    def splitDataByTime(self,dataset):
...

    def splitDataByTime(self,dataset):
...

    def computePop(self, dataset):
...

    def preprocess(self,trainingSet,testSet):
...
#remove items appearing in the training set from the test set
...

    def printTrainingSize(self):
...

    def getId(self,obj,t):
...

    def getSize(self,t):

#coding:utf8
...
class CDAE(IterativeRecommender):
...

    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):

    def readConfiguration(self):
...

    def initModel(self):
...

    def encoder(self,x,v):
...

    def decoder(self,x):
...

    def row(self, u):
...

    def next_batch(self):
...

    def buildModel(self):
...
# self.ranking_performance()

class Yue(object):
...

    def __init__(self,config):
...
# training data
# testData
...
#specify testSet
...
#auto partition
...
#cross validation
...
#self.trainingData,self.testData = DataSplit.crossValidation(self.trainingData,int(self.evaluation['-cv']))
...

    def execute(self):
...
#import the algorithm module
...
#create the manager used to communication in multiprocess
...
# if self.config.contains('social'):
#     recommender = self.config['recommender'] + "(self.config,train,test,self.relation,fold)"
# else:
...
#create the process
...
#start the processes
...
#wait until all processes are completed
...
#compute the mean error of k-fold cross validation
...
#output result

class IterativeRecommender(Recommender):
...

    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
...

    def readConfiguration(self):
...
# set the reduced dimension
...
# set maximum iteration
...
# set learning rate
...
# regularization parameter
...

    def printAlgorConfig(self):
...

    def initModel(self):
...
# latent user matrix
# latent item matrix
...

    def saveModel(self):
...

    def loadModel(self):
...

    def updateLearningRate(self,iter):
...

    def predict(self,u):

    def isConverged(self,iter):
...
#check if converged
...

    def evalRanking(self):
...
# used to contain the text of the result
...
# predict
...
# itemSet = sorted(itemSet.iteritems(), key=lambda d: d[1], reverse=True)
# if bTopN:
# find the K biggest scores
...
# ind = bisect(recommendations, itemSet[item])
...
# output prediction result
...
# output evaluation result
...

    def ranking_performance(self):
...
# find the K biggest scores

class Measure(object):
...

    def __init__(self):
...

    def hits(origin,res):
...

    def rankingMeasure(origin,res,N,itemCount):
...
#AUC = Measure.AUC(origin,res,rawRes)
#measure.append('AUC:' + str(AUC) + '\n')
...

    def coverage(res,itemCount):
...

    def precision(hits,N):
...

    def MAP(origin, res, N):
...

    def AUC(origin,res,rawRes):
...

    def recall(hits,origin):

#coding:utf8
...
class DHCF(DeepRecommender):
...

    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
...
# Dual Channel Hypergraph Collaborative Filtering (KDD 2020).
...

    def buildAdjacencyMatrix(self):
...

    def initModel(self):
...
# Build adjacency matrix
...
# Build incidence matrix
# H_u = hstack([A, A.dot(A.transpose().dot(A))])
...
# Build network
...

        def without_dropout(embedding):
...

        def dropout(embedding):
...
# user_embeddings = tf.reduce_sum(all_user_embeddings,axis=0)/(1+self.n_layer)
# item_embeddings = tf.reduce_sum(all_item_embeddings, axis=0) / (1 + self.n_layer)
#
...

    def next_batch_pairwise(self):
...
#negative sampling
...

    def buildModel(self):

#coding:utf8
...
# 先找到所有数据，构建词袋
# 构建网络，向网络中添加节点
# 向网络中的两个节点之间添加边的关系，生成路径
# 根据词袋，以及训练集，测试集，获得content和标签
#
...
class Dataset(object):
...

    def __init__(self, datanet, vocab_size, num_category, num_timesteps):
...

    def _parse_data(self, datanet):
...
# 用vocab_size填充
...
# 转换操作，将列表转换成numpy形式
...
# 进行随机化
# self._random_shuffle()
...

    def label_one_hot(self, label_id):
...

    def _random_shuffle(self):
...
# 随机化函数
...

    def next_batch(self, batch_size):
...
# batch_size 大于样本总数，提示异常
...

    def num_examples(self):
...
class ABLAH(IterativeRecommender):
...

    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
...

    def readConfiguration(self):
...
# the length of sequence: the number of nodes in every path
...

    def initModel(self):
...
# latent user matrix
# latent item matrix
...
# LSTM的步长，对齐minibatch
...
# LSTM梯度大小,防止梯度爆炸
...

    def ListenData(self):
...

    def _attention(self, H):
...
# 获得最后一层LSTM的神经元数量
...
# 初始化一个权重向量，是可训练的参数
...
# 对Bi-LSTM的输出用激活函数做非线性转换
...
# 对W和M做矩阵运算，W=[batch_size, time_step, hidden_size]，计算前做维度转换成[batch_size * time_step, hidden_size]
# newM = [batch_size, time_step, 1]，每一个时间步的输出由向量转换成一个数字
...
# 对newM做维度转换成[batch_size, time_step]
...
# 用softmax做归一化处理[batch_size, time_step]
...
# 利用求得的alpha的值对H进行加权求和，用矩阵运算直接操作
...
# 将三维压缩成二维sequeezeR=[batch_size, hidden_size]
...
# 对Attention的输出可以做dropout处理
...

    def _Bi_LSTMAttention(self, embeddedWords):
...
# 采用动态rnn，可以动态的输入序列的长度，若没有输入，则取序列的全长
# outputs是一个元祖(output_fw, output_bw)，其中两个元素的维度都是[batch_size, max_time, hidden_size],fw和bw的hidden_size一样
# self.current_state 是最终的状态，二元组(state_fw, state_bw)，state_fw=[batch_size, s]，s是一个元祖(h, c)
...
# 在Bi-LSTM+Attention的论文中，将前向和后向的输出相加
...
# 得到Attention的输出
...
# 全连接层的输出
...

    def _normalize(self, wordEmbedding, weights):
...

    def _addPerturbation(self, embedded, loss):
...

    def _scaleL2(self, x, norm_length):
...
# shape(x) = (batch, num_timesteps, d)
# Divide x by max(abs(x)) for a numerically stable L2 norm.
# 2norm(x) = a * 2norm(x/a)
# Scale over the full sequence, dims (1, 2)
...

    def buildModel(self):
...
# 加入所有数据，包括：用户、歌曲、艺人、专辑等，作为网络中的节点
...
# print('result_item:', result_item[-1])
# print('result_artist:', result_artist)
...
# self.walks = self.walks+list(nx.all_simple_paths(G, source=uid, target=tid, cutoff=self.cutoff))
...
#######################################################################################
...
# 保存训练的步数
...
# with tf.name_scope("embedding"):
...
# 初始化embedding
...
# [1, 10, 7] -> [embeddings[1], embeddings[10], embeddings[7]]
...
# # 利用词频计算新的词嵌入矩阵
# normWordEmbedding = self._normalize(tf.cast(wordEmbedding, dtype=tf.float32, name='word2vec'), weights)
...
# # 利用词嵌入矩阵将输入的数据中的词转换成词向量，维度[batch_size, sequence_length, embedding_size]
# self.embeddedWords = tf.nn.embedding_lookup(normWordEmbedding, self.inputs)
...
# perturPredictions = tf.argmax(perturPredictions, axis=1, name='perturPredictions')
...
# 定义优化函数， 传入学习率参数
...
# 计算梯度，得到梯度和变量
...
# 将梯度应用到变量下，生成训练器
...
# gradSummaries = []
# for g, v in self.gradsAndVars:
#     if g is not None:
#         tf.summary.histogram("{}/grad/hist".format(v.name), g)
#         tf.summary.scalar("{}/grad/sparsity".format(v.name), tf.nn.zero_fraction(g))
...
# outDir = os.path.abspath(os.path.join(os.path.curdir, "summarys"))
# print("Writing to {}\n".format(outDir))
...
# lossSummary = tf.summary.scalar("loss", self.loss)
# summaryOp = tf.summary.merge_all()

class CUNE(IterativeRecommender):
...

    def __init__(self,conf,trainingSet=None,testSet=None,fold='[1]'):
...

    def readConfiguration(self):
...

    def printAlgorConfig(self):
...

    def buildModel(self):
...
#build C-U-NET
...
#break infinite loop
...
#Training get top-k friends
...
# global user preference
...
#print self.topKSim
...
#prepare Pu set, IPu set, and Nu set
...
#self.NegativeSet = defaultdict(list)
...
# if len(self.NegativeSet[user])>0:
#     item_j = choice(self.NegativeSet[user])
# else:


yue.py, base/IterativeRecommender.py, data/record.py, evaluation/measure.py, recommender/advanced/ABLAH.py, pearson_sp, cosine_sp, Yue, ABLAH, CUNE, x2p, common, Hbeta, eig, real## tasks:
#collaborative filtering, general recommendation, general reinforcement learning,


This repository tackles the problem of collaborative filtering using the C-U-NET algorithm, which is an extension of the User-based and Item-based collaborative filtering models. It uses a sparse matrix representation of data from a CSV file.


Yue

**Founder**: [@Coder-Yu ](https://github.com/Coder-Yu)

More implementations of generic recommenders can be found in [RecQ](https://github.com/Coder-Yu)

Introduction

**Yue** is a Python library for Music Recommendation (Python 3.5.x). It implements a suit of state-of-the-art music recommenders. To run Yue easily (no need to setup dendencies used in RecQ one by one), the leading open data science platform  [**Anaconda**](https://www.continuum.io/downloads) is strongly recommended. It integrates Python interpreter, common scientific computing libraries (such as Numpy, Pandas, and Matplotlib), and package manager, all of them make it a perfect tool for data science researcher.

Features

Cross-platform: as a Python software, Yue can be easily deployed and executed in any platforms, including MS Windows, Linux and Mac OS.

Fast execution: Yue is based on the fast scientific computing libraries such as Numpy and some light common data structures, which make it run much faster than other libraries based on Python.

Easy configuration: Yue configs recommenders using a configuration file.

Easy expansion: Yue provides a set of well-designed recommendation interfaces by which new algorithms can be easily implemented.

How to Run it

1. Download the dataset. (links can be found at the end)

1. Configure the **xx.conf** file in the directory named config. (xx is the name of the algorithm you want to run)

2. Run the **main.py** in the project, and then input following the prompt.

How to Configure it

Essential Options

     Entry
    Example
    Description

    record
    D:/xiami/100K.txt
    Set the path to input dataset.

    record.setup
    -columns user:0,track:1,artist:2,album:3 -delim ,
    -columns: this option specifies what colums in the dataset mean. Four types of entities supported. If some types of information are missing, just skip the corresponding type;
 -delim: this option specifies which symbol separates the columns.

    recommender
    UserKNN/ItemKNN/MostPop/etc.
    the name of the recommender

    evaluation.setup
    -testSet ../dataset/testset.txt
    Main option: -testSet, -ap, -cv -byTime

      -testSet path/to/test/file   (need to specify the test set manually)

      -ap ratio   (ap means that the ratings are automatically partitioned into training set and test set, the number is the ratio of test set. e.g. -ap 0.2)

      -cv k   (-cv means cross validation, k is the number of the fold. e.g. -cv 5)

      -byTime ratio  (sort the user record in order of the time. ratio decides the percentage of test set(recently played).

      Secondary option:-b, -p, -cold

      -target track (This option decides which type of object will be recommended (artist, track, album). Only available for some general recommenders like MostPop)

      -b val ï¼binarizing the rating values. Ratings equal or greater than val will be changed into 1, and ratings lower than val will be changed into 0. e.g. -b 3.0ï¼

      -p (if this option is added, the cross validation wll be excuted parallelly, otherwise excuted one by one)

      -cold threshold (evaluation on cold-start users, users in training set with ratings more than threshold will be removed from the test set)

    item.ranking
    off -topN 5,10,20
    -topN N1,N2,N3...: the length of the recommendation list. *Yue can generate multiple evaluation results for different N at the same time

    output.setup
    on -dir ./Results/
    Main option: whether to output recommendation results

      -dir path: the directory path of output results.

Model-based Options

    num.factors
    5/10/20/number
    Set the number of latent factors

    num.max.iter
    100/200/number
    Set the maximum number of iterations for iterative recommendation algorithms.

    learnRate
    -init 0.01 -max 1
    -init initial learning rate for iterative recommendation algorithms;

      -max: maximum learning rate (default 1);

    reg.lambda
    -u 0.05 -i 0.05 -b 0.1

      -u: user regularizaiton; -i: item regularization; -b: bias regularizaiton;

How to extend it

1.Make your new algorithm generalize the proper base class.

2.Rewrite some of the following functions as needed.

           - readConfiguration()

           - printAlgorConfig()

           - initModel()

           - buildModel()

           - saveModel()

           - loadModel()

           - predict()

Algorithms Implemented

Note: We use SGD to obtain the local minimum. So, there have some differences between the original papers and the code in terms of fomula presentation. If you have problems in understanding the code, please open an issue to ask for help. We can guarantee that all the implementations are carefully reviewed and tested.

		Item Ranking
		Paper

    Rand
    Recommend tracks, artists or albums randomly

    MostPop
    Recommend most popular tracks, artists or albums

    UserKNN
    A common user-based collaborative filtering method

    BPR
    Rendle et al., BPR: Bayesian Personalized Ranking from Implicit Feedback, UAI 2009.

    WRMF
    Hu et al., Collaborative Filtering for Implicit Feedback Datasets, KDD 2009.

    IPF
    Xiang et al., Temporal Recommendation on Graphs via Long- and Short-term Preference Fusion, KDD 2010.

    FISM
    Kabbur et al., FISM: Factored Item Similarity Models for Top-N Recommender Systems, KDD 2013.

  ExpoMF
    Liang et al., Modeling User Exposure in Recommendation, WWW 2016.

    CDAE
    Wu et al., Collaborative Denoising Auto-Encoders for Top-N Recommender Systems, WSDM 2016.

    CUNE
    Zhang et al., Collaborative User Network Embedding for Social Recommender Systems, SDM 2017.

    Song2vec
    Cheng et al., Exploiting Music Play Sequence for Music Recommendation, IJCAI 2017

    APR
    He et al., Adversarial Personalized Ranking for Recommendation, SIGIR 2018.

Dataset

    Data Set
    Basic Meta
    Context

    Users
    Tracks
    Artists
    Albums
   Record
    Tag
    User Profile
   Artist Profile

    NowPlaying [1]
    1,744
    16,864
    2,108
   N/A
    1,117,335
    N/A
    N/A
    N/A

    Xiami [2]
    4,270
    177,289
    25,844
   68,479
    1,337,948
    N/A
    N/A
    N/A

    LastFM [3]
    1,892
    N/A
    17,632
    N/A
    92,834
    Yes
    N/A
    N/A

    Yahoo Music
	    [source]
    1,800,000
    136,000
    many
    many
    717,000,000
    Yes
    N/A
    N/A

    30 Music [source][4]
    45,167
    5,023,108
    595,049
   217,337
    many
    Yes
    Yes
    N/A

    Ali Music
    349,946
    10,278
    50
   N/A
    5,652,232
    N/A
    N/A
    Yes

 Dataset Reference

 [1]. Eva Zangerle, Martin Pichl, Wolfgang Gassler, and GÃ¼nther Specht. 2014. #nowplaying Music Dataset: Extracting Listening Behavior from Twitter. In Proceedings of the First International Workshop on Internet-Scale Multimedia Management (WISMM '14). ACM, New York, NY, USA, 21-26

 [2]. Wang, Dongjing, et al. "Learning music embedding with metadata for context aware recommendation." Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. ACM, 2016.

 [3]. IvÃ¡n Cantador, Peter Brusilovsky, and Tsvi Kuflik. 2011. 2nd Workshop on Information Heterogeneity and Fusion in Recom- mender Systems (HetRec 2011). In Proceedings of the 5th ACM conference on Recommender systems (RecSys 2011). ACM, New York, NY, USA

 [4]. Turrin R, Quadrana M, Condorelli A, et al. 30Music Listening and Playlists Dataset[C]//RecSys Posters. 2015.


datasets/convert_quiz.py:
⋮...
│def run(dataset_dir):
⋮...

datasets/dataset_utils.py:
⋮...
│def int64_feature(values):
⋮...
│def bytes_feature(values):
⋮...
│def image_to_tfexample(image_data, image_format, height, width, class_id):
⋮...
│def write_label_file(labels_to_class_names, dataset_dir,
⋮...

datasets/download_and_convert_cifar10.py:
⋮...
│def run(dataset_dir):
⋮...

datasets/download_and_convert_flowers.py:
⋮...
│def run(dataset_dir):
⋮...

datasets/download_and_convert_mnist.py:
⋮...
│def run(dataset_dir):
⋮...

deployment/model_deploy.py:
⋮...
│class DeploymentConfig(object):
│  """Configuration for deploying a model with `deploy()`.
│
│  You can pass an instance of this class to `deploy()` to specify exactly
│  how to deploy the model to build.  If you do not pass one, an instance built
│  from the default deployment_hparams will be used.
⋮...
│  def variables_device(self):
⋮...

nets/cyclegan.py:
⋮...
│def _dynamic_or_static_shape(tensor):
⋮...

nets/dcgan.py:
⋮...
│def _validate_image_inputs(inputs):
⋮...

nets/densenet.py:
⋮...
│def trunc_normal(stddev): return tf.truncated_normal_initializer(stddev=stddev)
│
⋮...
│def bn_act_conv_drp(current, num_outputs, kernel_size, scope='block'):
⋮...
│def block(net, layers, growth, scope='block'):
⋮...
│def transation(net,num_outputs,scope="transition"):
⋮...
│def densenet(images, num_classes=1001, is_training=False,
│             dropout_keep_prob=0.8,
│             scope='densenet'):
│    """Creates a variant of the densenet model.
│
│      images: A batch of `Tensors` of size [batch_size, height, width, channels].
│      num_classes: the number of classes in the dataset.
│      is_training: specifies whether or not we're currently training the model.
│        This variable will determine the behaviour of the dropout layer.
│      dropout_keep_prob: the percentage of activation values that are retained.
│      prediction_fn: a function to get predictions out of logits.
│      scope: Optional variable_scope.
│
⋮...
│    def reduce_dim(input_feature):
⋮...
│def bn_drp_scope(is_training=True, keep_prob=0.8):
⋮...

nets/inception_v4.py:
⋮...
│def inception_v4_base(inputs, final_endpoint='Mixed_7d', scope=None):
│  """Creates the Inception V4 network up to the given final endpoint.
│
│  Args:
│    inputs: a 4-D tensor of size [batch_size, height, width, 3].
│    final_endpoint: specifies the endpoint to construct the network up to.
│      It can be one of [ 'Conv2d_1a_3x3', 'Conv2d_2a_3x3', 'Conv2d_2b_3x3',
│      'Mixed_3a', 'Mixed_4a', 'Mixed_5a', 'Mixed_5b', 'Mixed_5c', 'Mixed_5d',
│      'Mixed_5e', 'Mixed_6a', 'Mixed_6b', 'Mixed_6c', 'Mixed_6d', 'Mixed_6e',
│      'Mixed_6f', 'Mixed_6g', 'Mixed_6h', 'Mixed_7a', 'Mixed_7b', 'Mixed_7c',
│      'Mixed_7d']
⋮...
│  def add_and_check_final(name, net):
⋮...

nets/mobilenet_v1.py:
⋮...
│def wrapped_partial(func, *args, **kwargs):
⋮...

preprocessing/inception_preprocessing.py:
⋮...
│def preprocess_for_train(image, height, width, bbox,
│                         fast_mode=True,
│                         scope=None,
⋮...
│def preprocess_for_eval(image, height, width,
⋮...

preprocessing/vgg_preprocessing.py:
⋮...
│def preprocess_for_train(image,
│                         output_height,
│                         output_width,
│                         resize_side_min=_RESIZE_SIDE_MIN,
⋮...
│def preprocess_for_eval(image, output_height, output_width, resize_side):
⋮...

train_eval_image_classifier.py:
⋮...
│def parse_args(check=True):
⋮...


# ç®ä»
æ¬ä»£ç ä¸ºç³»åè¯¾ç¨, ç¬¬ä¸å¨é¨åçè¯¾åä½ä¸åå®¹ã
https://edu.csdn.net/topic/ai115

å¦åèªå·±å®ç°ä¸ä¸ªdensenetçç½ç»ï¼å¹¶æå¥å°slimæ¡æ¶ä¸è¿è¡è®ç»ã

tinymind ä½¿ç¨è¯´æï¼https://gitee.com/ai100/quiz-w7-doc

### æ°æ®é
æ¬æ°æ®éæ¥æ200ä¸ªåç±»ï¼æ¯ä¸ªåç±»300å¼ å¾çï¼å±è®¡6Wå¼ å¾çï¼å¶ä¸5Wå¼ ä½ä¸ºè®ç»éï¼1Wå¼ å¾çä½ä¸ºéªè¯éãå¾çå·²ç»é¢æåä¸ºtfrecordæ ¼å¼å¹¶ä¸ä¼ å°tinymindä¸ãå°åå¦ä¸ï¼
https://www.tinymind.com/ai100/datasets/quiz-w7

### æ¨¡å
æ¨¡åä»£ç æ¥èªï¼
https://github.com/tensorflow/models/tree/master/research/slim

è¿éä¸ºäºéåºæ¬ä½ä¸æä¾çæ°æ®éï¼ç¨ä½ä¿®æ¹ï¼æ·»å äºä¸ä¸ªquizæ°æ®éä»¥åä¸ä¸ªè®ç»å¹¶éªè¯çèæ¬ï¼å®éä½¿ç¨çä»£ç ä¸ºï¼
https://gitee.com/ai100/quiz-w7-2-densenet

å¶ä¸netsç®å½ä¸çdensenet.pyä¸å·²ç»å®ä¹äºdensenetç½ç»çå¥å£å½æ°çï¼ç¸åºçè¾å©ä»£ç ä¹é½å·²ç»å®æï¼å¦ååªéè¦checkæèforkè¿éçä»£ç ï¼æ·»å èªå·±çdensenetå®ç°å¹¶å¨tinymindä¸å»ºç«ç¸åºçæ¨¡åå³å¯ã

densenetè®ºæåè https://arxiv.org/abs/1608.06993

å¨tinymindä¸æ°å»ºä¸ä¸ªæ¨¡åï¼æ¨¡åè®¾ç½®åèå¦ä¸æ¨¡åï¼

https://www.tinymind.com/ai100/quiz-w7-2-densenet
å¤å¶æ¨¡ååå¯ä»¥çå°æ¨¡åçå¨é¨åæ°ã

æ¨¡ååæ°çè§£éï¼

- dataset_name quiz  # æ°æ®éçåç§°ï¼è¿éä½¿ç¨æä»¬ä¸ºæ¬æ¬¡ä½ä¸ä¸é¨åçquizæ°æ®é
- dataset_dir /data/ai100/quiz-w7  # tfrecordåæ¾çç®å½ï¼è¿ä¸ªç®å½æ¯å»ºç«æ¨¡åçæ¶åï¼ç±tinymindæä¾ç
- model_name densenet  # ä½¿ç¨çç½ç»çåç§°ï¼æ¬ä½ä¸åºå®ä¸ºdensenet
- train_dir /output/ckpt  # è®ç»ç®å½ï¼è®ç»çä¸é´æä»¶åsummaryï¼checkpointçé½åæ¾å¨è¿éï¼è¿ä¸ªç®å½ä¹æ¯éªè¯è¿ç¨çcheckpoint_pathåæ°ï¼ è¿ä¸ªç®å½ç±tinymindæä¾ï¼éè¦æ³¨æè¿ä¸ªç®å½æ¯éè¦åå¥çï¼ä½¿ç¨å¶ä»ç®å½å¯è½ä¼åºç°åå¥å¤±è´¥çæåµã
- learning_rate 0.1  # å¦ä¹ ç, å ä¸ºæ²¡æé¢è®ç»æ¨¡åï¼è¿éä½¿ç¨è¾å¤§çå¦ä¹ çä»¥å å¿«æ¶æéåº¦ã
- optimizer rmsprop  # ä¼åå¨ï¼å³äºä¼åå¨çåºå«è¯·åè[è¿é](https://arxiv.org/abs/1609.04747)
- dataset_split_name validation # æ°æ®éåååï¼ç¨äºéªè¯è¿ç¨ï¼ä¼ å¥trainå¯éªè¯trainéåç¡®åº¦ï¼ä¼ å¥validationå¯éªè¯validationéåç¡®åº¦ï¼è¿éåªå³æ³¨validation
- eval_dir /output/eval  # éªè¯ç®å½ï¼éªè¯ç»æï¼åæ¬summaryçï¼ä¼åå¥è¿ä¸ªç®å½
- max_num_batches 128  # éªè¯batchesï¼è¿éä¼éªè¯128Ã32å±4096ä¸ªå¾çæ ·æ¬çæ°æ®ã

é¼å±åä¸è¯¾ç¨çå¦åå°è¯ä¸åçåæ°ç»åä»¥ä½éªä¸åçåæ°å¯¹è®ç»åç¡®çåæ¶æéåº¦çå½±åã

### ç»æè¯ä¼°
å¦åéè¦æä¾è¿è¡logçæªå¾åææ¡£æè¿°

å¨tinymindè¿è¡logçè¾åºä¸ï¼å¯ä»¥çå°å¦ä¸åå®¹ï¼
```sh
2017-12-1 23:03:04.327009: I tensorflow/core/kernels/logging_ops.cc:79] eval/Accuracy[0.252197266]
2017-12-1 23:03:04.327097: I tensorflow/core/kernels/logging_ops.cc:79] eval/Recall_5[0.494873047]
```
densenetçç½ç»ï¼ææè¦ç¥å¥½äºinceptionv4ãèèå°å®ç°çä¸åï¼èä¸æ²¡æé¢è®ç»æ¨¡åï¼è¿éä¸å¯¹åç¡®çåè¦æ±ãåªè¦è®ç»è¿è¡æåå¹¶æåç¡®çè¾åºå³å¯è®¤ä¸ºåæ ¼60åã

æä¾å¯¹densenetå®ç°è¿ç¨çæè¿°ï¼
å¯¹growthççè§£ 20å
å¯¹ç¨ å¯é¾æ¥ççè§£ 20å

# åèåå®¹
>epochè®¡ç®æ¹å¼ï¼
>epoch = step * batch_size / count_all_train_pics

æ¬å°è¿è¡slimæ¡æ¶æç¨å½ä»¤è¡ï¼

ä½¿ç¨é¢è®ç»æ¨¡åè¿è¡inceptionv4ççfinetune
```sh
è®ç»ï¼
python3 train_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --checkpoint_path=/path/to/inception_v4.ckpt --model_name=inception_v4 --checkpoint_exclude_scopes=InceptionV4/Logits,InceptionV4/AuxLogits/Aux_logits --train_dir=/path/to/train_ckpt --learning_rate=0.001 --optimizer=rmsprop  --batch_size=32

trainééªè¯ï¼
python3 eval_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --dataset_split_name=train --model_name=inception_v4 --checkpoint_path=/path/to/train_ckpt --eval_dir=/path/to/train_eval --batch_size=32 --max_num_batches=128

validationééªè¯ï¼
python3 eval_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --dataset_split_name=validation --model_name=inception_v4 --checkpoint_path=/path/to/train_ckpt --eval_dir=/path/to/validation_eval --batch_size=32 --max_num_batches=128

ç»ä¸èæ¬ï¼
python3 train_eval_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --checkpoint_path=/path/to/inception_v4.ckpt --model_name=inception_v4 --checkpoint_exclude_scopes=InceptionV4/Logits,InceptionV4/AuxLogits/Aux_logits --optimizer=rmsprop --train_dir=/path/to/log/train_ckpt --learning_rate=0.001 --dataset_split_name=validation --eval_dir=/path/to/eval --max_num_batches=128
```

ä»å¤´å¼å§è®ç»densenet
```sh
è®ç»
python3 train_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --model_name=densenet --train_dir=/path/to/train_ckpt_den --learning_rate=0.1 --optimizer=rmsprop  --batch_size=16/path/to

trainééªè¯ï¼
python3 eval_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --dataset_split_name=train --model_name=densenet --checkpoint_path=/path/to/train_ckpt_den --eval_dir=/path/to/train_eval_den --batch_size=32 --max_num_batches=128

validationééªè¯ï¼
python3 eval_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --dataset_split_name=validation --model_name=densenet --checkpoint_path=/path/to/train_ckpt_den --eval_dir=/path/to/validation_eval_den --batch_size=32 --max_num_batches=128

ç»ä¸èæ¬ï¼
python3 train_eval_image_classifier.py --dataset_name=quiz --dataset_dir=/path/to/data --model_name=densenet --checkpoint_exclude_scopes=InceptionV4/Logits,InceptionV4/AuxLogits/Aux_logits --train_dir=/path/to/log/train_ckpt --learning_rate=0.1 --dataset_split_name=validation --eval_dir=/path/to/eval_den --max_num_batches=128
```

## cpuè®ç»
æ¬å°æ²¡ææ¾å¡çæåµä¸ï¼ä½¿ç¨ä¸è¿°å½ä»¤è¿è¡è®ç»ä¼å¯¼è´éè¯¯ãåªä½¿ç¨CPUè¿è¡è®ç»çè¯ï¼éè¦å¨è®ç»å½ä»¤æèç»ä¸èæ¬ä¸æ·»å **--clone_on_cpu=True**åæ°ãtinymindä¸åéè¦æ°å»ºä¸ä¸ª**clone_on_cpu**ç**bool**ç±»ååæ°å¹¶è®¾ç½®ä¸º**True**

speaker specific lip to speech synthesis


This repository tackles the problem of image classification using DenseNet as a deep neural network architecture. The data used in this repository comes from the ImageNet dataset, which is a large-scale image recognition benchmark that provides high-quality images for training and testing various computer vision models.


#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
class BoundingBox(object):
...
# Failed to find "index" occurrence of item.
...
# pylint: disable=broad-except
...
# pylint: enable=broad-except
...
# Grab the 'index' annotation.
...
# Some images contain bounding box annotations that
# extend outside of the supplied image. See, e.g.
# n03127925/n03127925_147.xml
# Additionally, for some bounding boxes, the min > max
# or the box is entirely outside of the image.
...
# Example: <...>/n06470073/n00141669_6790.xml
...
# Determine if the annotation is from an ImageNet Challenge label.
...
# Note: There is a slight bug in the bounding box annotation data.
# Many of the dog labels have the human label 'Scottish_deerhound'
# instead of the synset ID 'n02092002' in the bbox.label field. As a
# simple hack to overcome this issue, we only exclude bbox labels
# *which are synset ID's* that do not match original synset label for
# the XML file.
...
# Guard against improperly specified boxes.

# Copyright 2016 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
# The URL where the Flowers data can be downloaded.
...
# The number of images in the validation set.
...
# Seed for repeatability.
...
# The number of shards per dataset split.
...
class ImageReader(object):
...

  def __init__(self):
...
# Initializes function that decodes RGB JPEG data.
...

  def read_image_dims(self, sess, image_data):

  def decode_jpeg(self, sess, image_data):
...
# Read the filename:
...
# Divide into train and test:
...
# First, convert the training and validation sets.

# Copyright 2016 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
# Seed for repeatability.
...
# The number of shards per dataset split.
...
class ImageReader(object):
...

    def __init__(self):
...
# Initializes function that decodes RGB JPEG data.
...

    def read_image_dims(self, sess, image_data):
...
#image = self.decode_jpeg(sess, image_data)
...

    def decode_jpeg(self, sess, image_data):
...
# Read the filename:
...
# Divide into train and test:
...
# First, convert the training and validation sets.

# Copyright 2016 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

# train
...
# eval
...
# train 1 epoch

# Copyright 2016 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
# The URL where the CIFAR data can be downloaded.
...
# The number of training files.
...
# The height and width of each image.
...
# The names of the classes.
...

    def _progress(count, block_size, total_size):
...
# First, process the training data:
...
# 1-indexed.
...
# Next, process the testing data:

# Copyright 2017 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
class ExportInferenceGraphTest(tf.test.TestCase):

#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
# Read in the 50000 synsets associated with the validation data set.
...
# Make all sub-directories in the validation data dir.

# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
...
# The labels file contains a list of valid labels are held in this file.
# Assumes that the file contains entries as such:
#   n01440764
#   n01443537
#   n01484850
# where each line corresponds to a label expressed as a synset. We map
# each synset contained in the file to an integer (based on the alphabetical
# ordering). See below for details.
...
# This file containing mapping from synset to human-readable label.
# Assumes each line of the file looks like:
#
#   n02119247    black fox
#   n02119359    silver fox
#   n02119477    red fox, Vulpes fulva
#
# where each line corresponds to a unique mapping. Note that each line is
# formatted as <synset>\t<human readable label>.
...
# This file is the output of process_bounding_box.py
# Assumes each line of the file looks like:
#
#   n00007846_64193.JPEG,0.0060,0.2620,0.7545,0.9940
#
# where each line corresponds to one bounding box annotation associated
# with an image. Each line can be parsed as:
#
#   <JPEG file name>, <xmin>, <ymin>, <xmax>, <ymax>
#
# Note that there might exist mulitple bounding box annotations associated
# with an image file.
...
# pylint: disable=expression-not-assigned
...
# pylint: enable=expression-not-assigned
...
class ImageCoder(object):
...

  def __init__(self):
...
# Create a single Session to run all image coding calls.
...
# Initializes function that converts PNG to JPEG data.
...
# Initializes function that converts CMYK JPEG data to RGB JPEG data.
...
# Initializes function that decodes RGB JPEG data.
...

  def png_to_jpeg(self, image_data):

  def cmyk_to_rgb(self, image_data):

  def decode_jpeg(self, image_data):
...
# File list from:
# https://groups.google.com/forum/embed/?place=forum/torch7#!topic/torch7/fOSTXHIESSU
...
# File list from:
# https://github.com/cytsai/ilsvrc-cmyk-image-list
...
# Read the image file.
...
# Clean the dirty data.
...
# 1 image is a PNG.
...
# 22 JPEG images are in CMYK colorspace.
...
# Decode the RGB JPEG.
...
# Check that image converted to RGB
...
# Each thread produces N shards where N = int(num_shards / num_threads).
# For instance, if num_shards = 128, and the num_threads = 2, then the first
# thread would produce shards [0, 64).
...
# Generate a sharded version of the file name, e.g. 'train-00002-of-00010'
...
# Break all images into batches with a [ranges[i][0], ranges[i][1]].
...
# Launch a thread for each batch.
...
# Create a mechanism for monitoring when all threads are finished.
...
# Create a generic TensorFlow-based utility for converting all image codings.
...
# Wait for all the threads to terminate.
...
# Leave label index 0 empty as a background class.
...
# Construct the list of JPEG files and labels.
...
# Shuffle the ordering of all image files in order to guarantee
# random ordering of the images with respect to label in the
# saved TFRecord files. Make the randomization repeatable.
...
# Build a map from synset to human-readable label.


Using summaries of '0492wzl/tensorflow_slim_densenet' files from Context, write repository README. Focus on the functionalities and features. There is no need to describe the dependencies and setup. The README should provide answers to the following questions:

* What machine learning problem does this repository tackle?
The repository tackles the problem of image classification using DenseNet as a deep neural network architecture.
* What kind of data does it use?
It uses images from the ImageNet dataset, which is a large-scale image recognition benchmark that provides high-quality images for training and testing various computer vision models.


1. `tf_record_creation_util.py`: This script creates TFRecord files from image files and labels. It takes a list of images and their corresponding labels as input, converts them to RGB format if necessary, and saves the converted images in TFRecord format. The script also generates sharded versions of the file names and breaks the images into batches with a specific range.
2. `inception_preprocessing.py`: This script provides a generic utility for converting all image codings to RGB format. It takes an image file as input, decodes it using JPEG decoding, checks that the image is in RGB format, and returns the converted image. The script also generates a sharded version of the file name and breaks the images into batches with a specific range.
3. `imagenet_data.py`: This script provides a mechanism for monitoring when all threads are finished. It takes a list of images and their corresponding labels as input, converts them to RGB format if necessary, and saves the converted images in TFRecord format. The script also generates sharded versions of the file names and breaks the images into batches with a specific range.
4. `


export_inference_graph_test.py, train_eval_image_classifier.py, datasets/convert_quiz.py, datasets/quiz.py, download_and_convert_data.py, transation, bn_drp_scope, densenet_arg_scope, trunc_normal, _large_imagenet_config, skip_reduction_layer_input, filter_scaling_rate, stem_cell, bounding_box_file, imagenet_metadata_file


export_inference_graph_test.py, train_eval_image_classifier.py, datasets/convert_quiz.py, datasets/quiz.py, download_and_convert_data.py, transation, bn_drp_scope, densenet_arg_scope, trunc_normal, _large_imagenet_config, skip_reduction_layer_input, filter_scaling_rate, stem_cell, bounding_box_file, imagenet_metadata_file## tasks:
#bert, embeddings, nlp),


bert, embeddings, nlp),


# Image Super-Resolution Using Deep Convolutional Networks

Tensorflow Implementation of SRCNN https://arxiv.org/pdf/1501.00092.pdf

super resolution
image super resolution
video super resolution


SOURCE/config.py

SOURCE/data.py:
⋮...
│def modcrop(image, scale=3):
⋮...
│def im2double(im):
⋮...
│class DATA():
│
│    def __init__(self, dirname):
│        self.dir_path = os.path.join(config.DATA_DIR, dirname)
│        self.filelist = os.listdir(self.dir_path)
│        self.size = len(self.filelist)
│        self.file_index = 0
│        self.data_index = 0
│        self.batch = None
⋮...
│    def read_img(self, filename):
⋮...
│    def preprocess_img(self, img):
⋮...
│    def process_img(self, file):
⋮...
│    def generate_batch( self):
⋮...

SOURCE/main.py

SOURCE/model.py:
⋮...
│class MODEL():
│
│    def __init__(self):
│        self.inputs = tf.placeholder(shape=[None, config.INPUT_SIZE, config.INPUT_SIZE, config.DIM]
│        self.labels = tf.placeholder(shape=[None, config.OUTPUT_SIZE, config.OUTPUT_SIZE, config.DI
│        self.logits = None
│        self.output = None
⋮...
│    def build(self):
⋮...
│    def train(self, data):
⋮...
│    def test(self, data):
⋮...
│    def sr_generate(self, data):
⋮...

SOURCE/neural_network.py:
⋮...
│class Layer():
│
│    def __init__(self, shape, mean, stddev):
│        self.weights = tf.Variable(tf.random_normal(shape=shape, mean=mean, stddev=stddev))
⋮...
│    def feed_forward(self, input_data, stride=None):
⋮...
│class Convolution_Layer(Layer):
│
│    def __init__(self, shape, mean, stddev):
⋮...
│    def feed_forward(self, input_data, stride):
⋮...
│class Output_Layer(Layer):
│
│    def __init__(self, shape, mean, stddev):
⋮...
│    def feed_forward(self, input_data, stride):
⋮...

SOURCE/utils.py:
⋮...
│def reconstruct(image, file):
⋮...
│def deprocess(imgs):
⋮...
│def stitch(patches):
⋮...


SOURCE/data.py, SOURCE/model.py, SOURCE/neural_network.py, SOURCE/utils.py, SOURCE/main.py, Convolution_Layer, Output_Layer, DataTests, im2double, stitch, SUB_IMG, NUM_SUBIMG, TEST_DIR, OUT_DIR, deprocess## tasks:
#data augmentation, super resolution, denoising, pipline,


SOURCE/data.py, SOURCE/model.py, SOURCE/neural_network.py, SOURCE/utils.py, SOURCE/main.py, Convolution_Layer, Output_Layer, DataTests, im2double, stitch, SUB_IMG, NUM_SUBIMG, TEST_DIR, OUT_DIR, deprocess


What machine learning problem does this repository tackle?
The repository tackles the problem of image resolution enhancement using a super-resolution convolutional neural network (SRCNN) model. The goal is to improve the resolution of low-resolution images.

What kind of data does it use?
The repository uses image data for training and testing the SRCNN model. The specific type of data used is not mentioned in the context, but based on the files provided, it appears to be a collection of images that can be read and processed using the `utils.py` file.


# -*- coding: utf-8 -*-
...
class MODEL():
...

    def __init__(self):
...

    def build(self):
...
#self.output = tf.image.resize_images(self.logits, [config.OUTPUT_SIZE, config.OUTPUT_SIZE], method=tf.image.ResizeMethod.BILINEAR)
...

    def train(self, data):
...

    def test(self, data):
...

    def sr_generate(self, data):

# -*- coding: utf-8 -*-
...
class DATA():
...

    def __init__(self, dirname):
...

    def read_img(self, filename):
...

    def preprocess_img(self, img):
...

    def process_img(self, file):
...

    def generate_batch( self):

# -*- coding: utf-8 -*-
...
#os.environ["CUDA_VISIBLE_DEVICES"]="1"
...
# READ DATA
...
# BUILD MODEL
...
# TRAIN MODEL
...
# TEST MODEL

# -*- coding: utf-8 -*-
...
class Layer():
...

    def __init__(self, shape, mean, stddev):

    def feed_forward(self, input_data, stride=None):
...
class Convolution_Layer(Layer):
...

    def __init__(self, shape, mean, stddev):
...

    def feed_forward(self, input_data, stride):
...
class Output_Layer(Layer):
...

    def __init__(self, shape, mean, stddev):

# -*- coding: utf-8 -*-
...
# DIRECTORY INFORMATION
...
# RANDOM NUMBER GENERATOR INFORMATION
...
# TRAINING INFORMATION
...
# IMAGE PROCESSING DETAILS


This repository tackles the problem of image resolution enhancement using a super-resolution convolutional neural network (SRCNN) model. The goal is to improve the resolution of low-resolution images. The repository uses image data for training and testing the SRCNN model.


data augmentation, super resolution, denoising, pipline,


* `utils.py`: This file contains utility functions for the project, such as reading and preprocessing images. It also defines the `DATA` class that is used to read and process image data.
* `model.py`: This file contains the implementation of the SRCNN model. The `MODEL` class defined in this file is responsible for building the model, training it, and testing it. The `build()` method is used to define the architecture of the model, while the `train()` and `test()` methods are used to train and test the model on image data.
* `data.py`: This file contains the implementation of the `DATA` class that is used to read and process image data. The `read_img()` method is used to read images from a directory, while the `preprocess_img()` method is used to preprocess the images before feeding them into the model.
* `main.py`: This file contains the main function of the project, which reads in the configuration file and calls the appropriate functions to build, train, and test the SRCNN model on image data.
* `neural_network.py`: This file contains the implementation of the neural network architecture used in


models/helpers.py:
⋮...
│def _go_frames(batch_size, output_dim):
⋮...
│def _teacher_forcing_ratio_decay(init_tfr, global_step):
⋮...

models/modules.py:
⋮...
│def prenet(inputs, is_training, layer_sizes, scope=None):
⋮...
│def cbhg(inputs, input_lengths, is_training, scope, K, projections, depth):
⋮...
│def highwaynet(inputs, scope, depth):
⋮...
│def conv1d(inputs, kernel_size, channels, activation, is_training, drop_rate, scope):
⋮...

models/rnn_wrappers.py:
⋮...
│class TacotronDecoderCellState(
│  collections.namedtuple("TacotronDecoderCellState",
│   ("cell_state", "attention", "time", "alignments",
│    "alignment_history"))):
│  """`namedtuple` storing the state of a `TacotronDecoderCell`.
│  Contains:
│    - `cell_state`: The state of the wrapped `RNNCell` at the previous time
│    step.
│    - `attention`: The attention emitted at the previous time step.
│    - `time`: int32 scalar containing the current time step.
│    - `alignments`: A single or tuple of `Tensor`(s) containing the alignments
│     emitted at the previous time step for each attention mechanism.
│    - `alignment_history`: a single or tuple of `TensorArray`(s)
│     containing alignment matrices from all time steps for each attention
⋮...
│  def replace(self, **kwargs):
⋮...
│class TacotronDecoderWrapper(RNNCell):
│  """Tactron 2 Decoder Cell
│  Decodes encoder output and previous mel frames into next r frames
│
│  Decoder Step i:
│    1) Prenet to compress last output information
│    2) Concat compressed inputs with previous context vector (input feeding) *
│    3) Decoder RNN (actual decoding) to predict current state s_{i} *
│    4) Compute new context vector c_{i} based on s_{i} and a cumulative sum of previous alignments
│    5) Predict new output y_{i} using s_{i} and c_{i} (concatenated)
│    6) Predict <stop_token> output ys_{i} using s_{i} and c_{i} (concatenated)
│
⋮...
│  def zero_state(self, batch_size, dtype):
⋮...

models/tacotron.py:
⋮...
│class Tacotron():
⋮...

synthesizer.py:
⋮...
│class Synthesizer:
│  def load(self, checkpoint_path, model_name='tacotron'):
│    print(('Constructing model: %s' % model_name))
│    inputs = tf.placeholder(tf.int32, [1, None], 'inputs')
│    input_lengths = tf.placeholder(tf.int32, [1], 'input_lengths')
│    with tf.variable_scope('model') as scope:
│      self.model = create_model(model_name, hparams)
│      self.model.initialize(inputs, input_lengths)
│      self.wav_output = audio.inv_spectrogram_tensorflow(self.model.linear_outputs[0])
│
│    print(('Loading checkpoint: %s' % checkpoint_path))
⋮...

text/cleaners.py:
⋮...
│def collapse_whitespace(text):
⋮...

text/numbers.py:
⋮...
│def normalize_numbers(text):
⋮...

util/__init__.py:
⋮...
│  def __init__(self, window_size=100):
│    self._window_size = window_size
⋮...
│  def append(self, x):
⋮...
│  @property
│  def sum(self):
⋮...

util/audio.py:
⋮...
│def preemphasis(x):
⋮...
│def inv_preemphasis(x):
⋮...

util/infolog.py:
⋮...
│def log(msg, slack=False):
⋮...
│def _close_logfile():
⋮...


1. `models.py`: This file contains the implementation of the Tacotron model architecture, including the encoder, decoder, postnet, and attention mechanisms. It also defines the loss function used for training the model.
2. `datafeeder.py`: This file implements a data feeder class that loads audio files from disk and prepares them for input to the Tacotron model. It also handles bucketing of examples based on their output sequence length for efficiency.
3. `thchs30.py`: This file contains code for loading the THCHS-30 dataset, which is a collection of Chinese speech data used in training and evaluation of the Tacotron model. The file defines functions for reading audio files from disk, rescaling them to a uniform measure, trimming silence, computing linear-scale spectrograms, and writing the spectrograms to disk.

In summary, these files implement the core machine learning models and data processing components of the Tacotron model, including the encoder-decoder architecture, attention mechanisms, loss function, and data loading and preprocessing functions for the THCHS-30 dataset.

text to speech synthesis
frame
speech synthesis


asr,


datasets/biaobei.py, datasets/thchs30.py, models/custom_decoder.py, datasets/datafeeder.py, tests/cmudict_test.py, preprocess_biaobei, _pad_stop_token_target, inv_spectrogram_tensorflow, _stft_tensorflow, _istft_tensorflow, preprocess_biaobei, preprocess_blizzard, _prepare_stop_token_targets, get_output_base_path, preprocess_ljspeech## tasks:
#asr,


datasets/biaobei.py, datasets/thchs30.py, models/custom_decoder.py, datasets/datafeeder.py, tests/cmudict_test.py, preprocess_biaobei, _pad_stop_token_target, inv_spectrogram_tensorflow, _stft_tensorflow, _istft_tensorflow, preprocess_biaobei, preprocess_blizzard, _prepare_stop_token_targets, get_output_base_path, preprocess_ljspeech


This repository tackles the problem of speech synthesis using a deep neural network architecture called Tacotron. It uses a combination of encoder-decoder and postnet models, as well as attention mechanisms to generate high-quality speech from text inputs. The repository also includes code for loading and preprocessing data from the THCHS-30 dataset, which is a collection of Chinese speech data used in training and evaluation of the Tacotron model.

The repository provides an implementation of the Tacotron model architecture, including the encoder, decoder, postnet, and attention mechanisms. It also defines the loss function used for training the model. Additionally, it includes a data feeder class that loads audio files from disk and prepares them for input to the Tacotron model, as well as functions for reading audio files from disk, rescaling them to a uniform measure, trimming silence, computing linear-scale spectrograms, and writing the spectrograms to disk.

Overall, this repository provides a comprehensive implementation of the Tacotron model architecture and its components, as well as code for loading and preprocessing data from the THCHS-30 dataset. It can be used for training and evaluating the model on various speech synthesis tasks.


class CustomDecoder(decoder.Decoder):
...
# To use layer's compute_output_shape, we need to convert the
# RNNCell's output_size entries into shapes with an unknown
# batch size.  We then pass this through the layer's
# compute_output_shape and read off all but the first (batch)
# dimensions to get the output size of the rnn with the layer
# applied to the top.
...
# pylint: disable=protected-access
...
# Return the cell output and the id
...
# Assume the dtype of the cell is the output_size structure
# containing the input_state's first component's dtype.
# Return that structure and the sample_ids_dtype from the helper.
...
#Call outputprojection wrapper cell

# We use ProcessPoolExecutor to parallize across processes. This is just an optimization and you
# can omit it and just call _process_utterance on each input if you want.
...
# Load the audio to a numpy array:
...
# rescale wav for unified measure for all clips
...
# trim silence
...
# Compute the linear-scale spectrogram from the wav:
...
# Compute a mel-scale spectrogram from the wav:
...
# Write the spectrograms to disk:

# We use ProcessPoolExecutor to parallize across processes. This is just an optimization and you
# can omit it and just call _process_utterance on each input if you want.
...
# Load the audio to a numpy array:
...
# Compute the linear-scale spectrogram from the wav:
...
# Compute a mel-scale spectrogram from the wav:
...
# Write the spectrograms to disk:

class FrameProjection:

  def __init__(self, shape=hp.num_mels, activation=None, scope=None):
...

  def __call__(self, inputs):
...
# If activation==None, this returns a simple Linear projection
# else the projection will be passed through an activation function
# output = tf.layers.dense(inputs, units=self.shape, activation=self.activation,
# name='projection_{}'.format(self.scope))
...
class StopProjection:

  def __init__(self, is_training, shape=1, activation=tf.nn.sigmoid, scope=None):
...

  def __call__(self, inputs):
...
#During training, don't use activation as it is integrated inside the sigmoid_cross_entropy loss function
...

  def replace(self, **kwargs):
...
class TacotronDecoderWrapper(RNNCell):
...

  def __init__(self, is_training, attention_mechanism, rnn_cell, frame_projection, stop_projection):
...
#Initialize decoder layers
...

  def _batch_size_checks(self, batch_size, error_message):
...

  def output_size(self):
...

  def state_size(self):
...
# @property
...

  def zero_state(self, batch_size, dtype):
...

  def __call__(self, inputs, state):
...
#Information bottleneck (essential for learning attention)
...
#Concat context vector and prenet output to form RNN cells input (input feeding)
...
#Unidirectional RNN layers
...
#Compute the attention (context) vector and alignments using
#the new decoder cell hidden state as query vector
#and cumulative alignments to extract location features
#The choice of the new cell hidden state (s_{i}) of the last
#decoder RNN Cell is based on Luong et Al. (2015):
#https://arxiv.org/pdf/1508.04025.pdf
...
#Concat RNN outputs and context vector to form projections inputs
...
#Compute predicted frames and predicted <stop_token>
...
#Save alignment history

class DataFeeder(threading.Thread):
...

  def __init__(self, coordinator, metadata_filename, hparams):
...
# Load metadata:
...
# Create placeholders for inputs and targets. Don't specify batch size because we want to
# be able to feed different sized batches at eval time.
...
# Create queue for buffering data:
...
# Load CMUDict: If enabled, this will randomly substitute some words in the training data with
# their ARPABet equivalents, which will allow you to also pass ARPABet to the model for
# synthesis (useful for proper nouns, etc.)
...

  def start_in_session(self, session):

  def run(self):
...

  def _enqueue_next_group(self):
...
# Read a group of examples:
...
# Bucket examples based on similar output sequence length for efficiency:
...

  def _get_next_example(self):

# We use ProcessPoolExecutor to parallize across processes. This is just an optimization and you
# can omit it and just call _process_utterance on each input if you want.
...
# Load the audio to a numpy array:
...
# rescale wav for unified measure for all clips
...
# trim silence
...
# Compute the linear-scale spectrogram from the wav:
...
# Compute a mel-scale spectrogram from the wav:
...
# Write the spectrograms to disk:


This repository tackles the problem of speech synthesis using a deep neural network architecture called Tacotron. It uses a combination of encoder-decoder and postnet models, as well as attention mechanisms to generate high-quality speech from text inputs. The repository also includes code for loading and preprocessing data from the THCHS-30 dataset, which is a collection of Chinese speech data used in training and evaluation of the Tacotron model.


# Tacotron

An implementation of Tacotron speech synthesis in TensorFlow.

### Audio Samples

  * **[Audio Samples](https://keithito.github.io/audio-samples/)** from models trained using this repo.
    * The first set was trained for 877K steps on the [LJ Speech Dataset](https://keithito.com/LJ-Speech-Dataset/)
      * Speech started to become intelligble around 20K steps.
      * Although loss continued to decrease, there wasn't much noticable improvement after ~250K steps.
    * The second set was trained by [@MXGray](https://github.com/MXGray) for 140K steps on the [Nancy Corpus](http://www.cstr.ed.ac.uk/projects/blizzard/2011/lessac_blizzard2011/).

## Background

In April 2017, Google published a paper, [Tacotron: Towards End-to-End Speech Synthesis](https://arxiv.org/pdf/1703.10135.pdf),
where they present a neural text-to-speech model that learns to synthesize speech directly from
(text, audio) pairs. However, they didn't release their source code or training data. This is an
independent attempt to provide an open-source implementation of the model described in their paper.

The quality isn't as good as Google's demo yet, but hopefully it will get there someday :-).
Pull requests are welcome!

## Quick Start

### Installing dependencies

1. Install Python 3.

2. Install the latest version of [TensorFlow](https://www.tensorflow.org/install/) for your platform. For better
   performance, install with GPU support if it's available. This code works with TensorFlow 1.3 and later.

3. Install requirements:
   ```
   pip install -r requirements.txt
   ```

### Using a pre-trained model

1. **Download and unpack a model**:
   ```
   curl http://data.keithito.com/data/speech/tacotron-20170720.tar.bz2 | tar xjC /tmp
   ```

2. **Run the demo server**:
   ```
   python3 demo_server.py --checkpoint /tmp/tacotron-20170720/model.ckpt
   ```

3. **Point your browser at localhost:9000**
   * Type what you want to synthesize

### Training

*Note: you need at least 40GB of free disk space to train a model.*

1. **Download a speech dataset.**

   The following are supported out of the box:
    * [LJ Speech](https://keithito.com/LJ-Speech-Dataset/) (Public Domain)
    * [Blizzard 2012](http://www.cstr.ed.ac.uk/projects/blizzard/2012/phase_one) (Creative Commons Attribution Share-Alike)

   You can use other datasets if you convert them to the right format. See [TRAINING_DATA.md](TRAINING_DATA.md) for more info.

2. **Unpack the dataset into `~/tacotron`**

   After unpacking, your tree should look like this for LJ Speech:
   ```
   tacotron
     |- LJSpeech-1.1
         |- metadata.csv
         |- wavs
   ```

   or like this for Blizzard 2012:
   ```
   tacotron
     |- Blizzard2012
         |- ATrampAbroad
         |   |- sentence_index.txt
         |   |- lab
         |   |- wav
         |- TheManThatCorruptedHadleyburg
             |- sentence_index.txt
             |- lab
             |- wav
   ```

3. **Preprocess the data**
   ```
   python3 preprocess.py --dataset ljspeech
   ```
     * Use `--dataset blizzard` for Blizzard data

4. **Train a model**
   ```
   python3 train.py
   ```

   Tunable hyperparameters are found in [hparams.py](hparams.py). You can adjust these at the command
   line using the `--hparams` flag, for example `--hparams="batch_size=16,outputs_per_step=2"`.
   Hyperparameters should generally be set to the same values at both training and eval time.
   The default hyperparameters are recommended for LJ Speech and other English-language data.
   See [TRAINING_DATA.md](TRAINING_DATA.md) for other languages.

5. **Monitor with Tensorboard** (optional)
   ```
   tensorboard --logdir ~/tacotron/logs-tacotron
   ```

   The trainer dumps audio and alignments every 1000 steps. You can find these in
   `~/tacotron/logs-tacotron`.

6. **Synthesize from a checkpoint**
   ```
   python3 demo_server.py --checkpoint ~/tacotron/logs-tacotron/model.ckpt-185000
   ```
   Replace "185000" with the checkpoint number that you want to use, then open a browser
   to `localhost:9000` and type what you want to speak. Alternately, you can
   run [eval.py](eval.py) at the command line:
   ```
   python3 eval.py --checkpoint ~/tacotron/logs-tacotron/model.ckpt-185000
   ```
   If you set the `--hparams` flag when training, set the same value here.

## Notes and Common Issues

  * [TCMalloc](http://goog-perftools.sourceforge.net/doc/tcmalloc.html) seems to improve
    training speed and avoids occasional slowdowns seen with the default allocator. You
    can enable it by installing it and setting `LD_PRELOAD=/usr/lib/libtcmalloc.so`. With TCMalloc,
    you can get around 1.1 sec/step on a GTX 1080Ti.

  * You can train with [CMUDict](http://www.speech.cs.cmu.edu/cgi-bin/cmudict) by downloading the
    dictionary to ~/tacotron/training and then passing the flag `--hparams="use_cmudict=True"` to
    train.py. This will allow you to pass ARPAbet phonemes enclosed in curly braces at eval
    time to force a particular pronunciation, e.g. `Turn left on {HH AW1 S S T AH0 N} Street.`

  * If you pass a Slack incoming webhook URL as the `--slack_url` flag to train.py, it will send
    you progress updates every 1000 steps.

  * Occasionally, you may see a spike in loss and the model will forget how to attend (the
    alignments will no longer make sense). Although it will recover eventually, it may
    save time to restart at a checkpoint prior to the spike by passing the
    `--restore_step=150000` flag to train.py (replacing 150000 with a step number prior to the
    spike). **Update**: a recent [fix](https://github.com/keithito/tacotron/pull/7) to gradient
    clipping by @candlewill may have fixed this.

  * During eval and training, audio length is limited to `max_iters * outputs_per_step * frame_shift_ms`
    milliseconds. With the defaults (max_iters=200, outputs_per_step=5, frame_shift_ms=12.5), this is
    12.5 seconds.

    If your training examples are longer, you will see an error like this:
    `Incompatible shapes: [32,1340,80] vs. [32,1000,80]`

    To fix this, you can set a larger value of `max_iters` by passing `--hparams="max_iters=300"` to
    train.py (replace "300" with a value based on how long your audio is and the formula above).

  * Here is the expected loss curve when training on LJ Speech with the default hyperparameters:
    ![Loss curve](https://user-images.githubusercontent.com/1945356/36077599-c0513e4a-0f21-11e8-8525-07347847720c.png)

## Other Implementations
  * By Alex Barron: https://github.com/barronalex/Tacotron
  * By Kyubyong Park: https://github.com/Kyubyong/tacotron


class SmilesEmbbeding(torch.nn.Module):
...
## 56
...

    def forward(self, x):
...
class CDN(torch.nn.Module):
...
# H, W
...
# tensorflow filter =  [filter_height, filter_width, in_channels, out_channels]
# TODO: tf.nn.embedding_lookup()
...

    def encode(self, x):
...
# x shape is B, W, H
...
# x shape is B, C=1, W, H
...

    def decode(self, z, x):
...
#decoder_inputs_list = embed_x_list
...

    def forward(self, x):
...

    def loss_acc(self, x, logits, latent_loss):

class SmilesDataLoader(object):
...

    def __init__(self, filename, max_length=50, smiles_chars=None):
...

    def _preprocess(self, raw_data):
...

    def _read(self, filename):
...

    def __getitem__(self, i):
...

    def __len__(self):


The repository '0h-n0/cdn_molecule_pytorch' tackles the problem of generating new molecules based on a given SMILES string using a deep generative model. The data used is SMILES strings, which are used as input to the CDN (Convolutional Denoising Network) for generating new molecules.

The repository provides an implementation of the CDN, which is a deep neural network architecture designed for molecular generation. It includes the encoder and decoder networks, which work together to generate new molecules from a given input. The `encode` method takes in a sequence of SMILES strings and outputs a latent representation of the molecule, while the `decode` method takes in the latent representation and generates a new SMILES string. The `forward` method combines these two methods to generate a new molecule from a given input.

The repository also includes code for loading and preprocessing SMILES strings, which are used as input to the CDN. This allows users to easily use the CDN to generate new molecules based on their own SMILES strings.


The `cdn` directory contains the implementation of a deep generative model for molecular generation, called the Convolutional Denoising Network (CDN). The `data` directory contains code for loading and preprocessing SMILES strings, which are used as input to the CDN. The `examples` directory contains an example script for using the CDN to generate new molecules based on a given SMILES string.

The `cdn/module.py` file defines the architecture of the CDN, including the encoder and decoder networks. The `encode` method takes in a sequence of SMILES strings and outputs a latent representation of the molecule, while the `decode` method takes in the latent representation and generates a new SMILES string. The `forward` method is used for training and inference, and it combines the `encode` and `decode` methods to generate a new molecule from a given input.

The `cdn/data.py` file defines a class for loading and preprocessing SMILES strings, which are used as input to the CDN. The `_preprocess` method tokenizes the SMILES string and adds special tokens to indicate the start and end


cdn/data.py, cdn/module.py, examples/example_zinc.py, cdn/util.py, cdn/__init__.py, SmilesEmbbeding, CDN, SmilesDataLoader, get_iterators, get_version, SmilesDataLoader, get_iterators, scatter_, resolve, urlretrieve


cdn/data.py, cdn/module.py, examples/example_zinc.py, cdn/util.py, cdn/__init__.py, SmilesEmbbeding, CDN, SmilesDataLoader, get_iterators, get_version, SmilesDataLoader, get_iterators, scatter_, resolve, urlretrieve## tasks:
#chemistry, power spectrum, molecular graph, megatron-lm,


chemistry, power spectrum, molecular graph, megatron-lm,


# CDN_molecule_pytorch

Pytorch Implementation https://arxiv.org/abs/1804.02668.

## Requirements

* Python >= 3.6
* Pytroch >= 0.4
* rdkit

## How to use

```
```

## Example

See examples/examle_zinc.py

## Original repository (Tensorflow implementation)

* https://github.com/shaharharel/CDN_Molecule


The '0h-n0/cdn_molecule_pytorch' repository tackles the problem of generating new molecules based on a given SMILES string using a deep generative model. The data used is SMILES strings, which are used as input to the CDN (Convolutional Denoising Network) for generating new molecules. The repository provides an implementation of the CDN, which is a deep neural network architecture designed for molecular generation. It includes the encoder and decoder networks, which work together to generate new molecules from a given input. The `encode` method takes in a sequence of SMILES strings and outputs a latent representation of the molecule, while the `decode` method takes in the latent representation and generates a new SMILES string. The `forward` method combines these two methods to generate a new molecule from a given input. The repository also includes code for loading and preprocessing SMILES strings, which are used as input to the CDN. This allows users to easily use the CDN to generate new molecules based on their own SMILES strings.


cdn/__init__.py

cdn/data.py:
⋮...
│class SmilesDataLoader(object):
│    _SMILES_CHARS = [' ',
│                     '#', '%', '(', ')', '+', '-', '.', '/',
│                     '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
│                     '=', '@',
│                     'A', 'B', 'C', 'F', 'H', 'I', 'K', 'L', 'M', 'N', 'O', 'P',
│                     'R', 'S', 'T', 'V', 'X', 'Z',
│                     '[', '\\', ']',
│                     'a', 'b', 'c', 'e', 'g', 'i', 'l', 'n', 'o', 'p', 'r', 's',
⋮...
│    def __init__(self, filename, max_length=50, smiles_chars=None):
⋮...
│    def _preprocess(self, raw_data):
⋮...
│    def _read(self, filename):
⋮...
│    def _smiles_to_vec(self, smiles: str, smiles_chars=None):
⋮...

cdn/module.py:
⋮...
│def one_hot(x, size):
⋮...
│class SmilesEmbbeding(torch.nn.Module):
│    _SMILES_CHARS = [' ',
│                     '#', '%', '(', ')', '+', '-', '.', '/',
│                     '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
│                     '=', '@',
│                     'A', 'B', 'C', 'F', 'H', 'I', 'K', 'L', 'M', 'N', 'O', 'P',
│                     'R', 'S', 'T', 'V', 'X', 'Z',
│                     '[', '\\', ']',
│                     'a', 'b', 'c', 'e', 'g', 'i', 'l', 'n', 'o', 'p', 'r', 's',
⋮...
│    def __init__(self,
│                 embedding_dim,
│                 smiles_chars=None,
│                 padding_idx=None,
│                 max_norm=None,
│                 norm_type=2,
│                 scale_grad_by_freq=False,
│                 sparse=False,
⋮...
│class CDN(torch.nn.Module):
│    def __init__(self,
│                 embedding_dim=128,
│                 kernel_sizes=[3, 4, 5, 6],
│                 in_channels=[1, 1, 1, 1],
│                 out_channels=[128, 128, 128, 128],
│                 max_mol_length=50,
│                 smiles_chars=None,
│                 variational=True,
│                 gaussian_samples=300,
│                 generation_mode=False,
⋮...
│    def encode(self, x):
⋮...
│    def decode(self, z, x):
⋮...
│    def loss_acc(self, x, logits, latent_loss):
⋮...

cdn/util.py

examples/example_zinc.py:
⋮...
│@ex.capture
│def download(_log, data_root, filenames):
⋮...
│@ex.capture
│def get_iterators(_log, data_root, filenames):
⋮...
│@ex.capture
│def train(_log):
⋮...

setup.py:
⋮...
│def get_version(package):
⋮...


The '000Justin000/torchdiffeq' repository is a PyTorch implementation of the ODE solver that allows for solving ordinary differential equations (ODEs) using neural networks. The machine learning problem that this repository tackles is the ability to approximate complex functions using neural networks. By using a neural network to represent the solution of an ODE, we can learn the underlying patterns in the data and make predictions about future values of the function. This is particularly useful when dealing with high-dimensional data or when the true solution of the ODE is difficult to obtain. The repository uses a variety of datasets from different sources, including the MNIST dataset for image classification and the CIFAR-10 dataset for object detection. The data used in these examples is typically preprocessed and normalized to have zero mean and unit variance, which helps the neural network learn more efficiently. Overall, the '000Justin000/torchdiffeq' repository provides a powerful tool for solving ODEs using neural networks, and it has the potential to be used in a wide range of applications, such as scientific computing, engineering, finance, and more.

multivariate time series forecasting
point processes
multivariate time series imputation
time series


The '000Justin000/torchdiffeq' repository is a PyTorch implementation of the ODE solver, which allows for solving ordinary differential equations (ODEs) using neural networks. The repository provides a simple and efficient way to solve ODEs by approximating the solution using a neural network.

The machine learning problem that this repository tackles is the ability to approximate complex functions using neural networks. By using a neural network to represent the solution of an ODE, we can learn the underlying patterns in the data and make predictions about future values of the function. This is particularly useful when dealing with high-dimensional data or when the true solution of the ODE is difficult to obtain.

The repository uses a variety of datasets from different sources, including the MNIST dataset for image classification and the CIFAR-10 dataset for object detection. The data used in these examples is typically preprocessed and normalized to have zero mean and unit variance, which helps the neural network learn more efficiently.

Overall, the '000Justin000/torchdiffeq' repository provides a powerful tool for solving ODEs using neural networks, and it has