Upload folder using huggingface_hub

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Chuan Guo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,232 @@
+---
+title: MoMask
+emoji: 🎭
+colorFrom: pink
+colorTo: purple
+sdk: gradio
+sdk_version: 3.24.1
+app_file: app.py
+pinned: false
+---
+
+# MoMask: Generative Masked Modeling of 3D Human Motions
+## [[Project Page]](https://ericguo5513.github.io/momask) [[Paper]](https://arxiv.org/abs/2312.00063)
+![teaser_image](https://ericguo5513.github.io/momask/static/images/teaser.png)
+
+If you find our code or paper helpful, please consider citing:
+```
+@article{guo2023momask,
+      title={MoMask: Generative Masked Modeling of 3D Human Motions}, 
+      author={Chuan Guo and Yuxuan Mu and Muhammad Gohar Javed and Sen Wang and Li Cheng},
+      year={2023},
+      eprint={2312.00063},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```
+
+## :postbox: News
+📢 **2023-12-19** --- Release scripts for temporal inpainting.
+
+📢 **2023-12-15** --- Release codes and models for momask. Including training/eval/generation scripts.
+
+📢 **2023-11-29** --- Initialized the webpage and git project.  
+
+
+## :round_pushpin: Get You Ready
+
+<details>
+  
+### 1. Conda Environment
+```
+conda env create -f environment.yml
+conda activate momask
+pip install git+https://github.com/openai/CLIP.git
+```
+We test our code on Python 3.7.13 and PyTorch 1.7.1
+
+
+### 2. Models and Dependencies
+
+#### Download Pre-trained Models
+```
+bash prepare/download_models.sh
+```
+
+#### Download Evaluation Models and Gloves
+For evaluation only.
+```
+bash prepare/download_evaluator.sh
+bash prepare/download_glove.sh
+```
+
+#### Troubleshooting
+To address the download error related to gdown: "Cannot retrieve the public link of the file. You may need to change the permission to 'Anyone with the link', or have had many accesses". A potential solution is to run `pip install --upgrade --no-cache-dir gdown`, as suggested on https://github.com/wkentaro/gdown/issues/43. This should help resolve the issue.
+
+#### (Optional) Download Mannually
+Visit [[Google Drive]](https://drive.google.com/drive/folders/1b3GnAbERH8jAoO5mdWgZhyxHB73n23sK?usp=drive_link) to download the models and evaluators mannually.
+
+### 3. Get Data
+
+You have two options here:
+* **Skip getting data**, if you just want to generate motions using *own* descriptions.
+* **Get full data**, if you want to *re-train* and *evaluate* the model.
+
+**(a). Full data (text + motion)**
+
+**HumanML3D** - Follow the instruction in [HumanML3D](https://github.com/EricGuo5513/HumanML3D.git), then copy the result dataset to our repository:
+```
+cp -r ../HumanML3D/HumanML3D ./dataset/HumanML3D
+```
+**KIT**-Download from [HumanML3D](https://github.com/EricGuo5513/HumanML3D.git), then place result in `./dataset/KIT-ML`
+
+#### 
+
+</details>
+
+## :rocket: Demo
+<details>
+
+### (a) Generate from a single prompt
+```
+python gen_t2m.py --gpu_id 1 --ext exp1 --text_prompt "A person is running on a treadmill."
+```
+### (b) Generate from a prompt file
+An example of prompt file is given in `./assets/text_prompt.txt`. Please follow the format of `<text description>#<motion length>` at each line. Motion length indicates the number of poses, which must be integeter and will be rounded by 4. In our work, motion is in 20 fps.
+
+If you write `<text description>#NA`, our model will determine a length. Note once there is **one** NA, all the others will be **NA** automatically.
+
+```
+python gen_t2m.py --gpu_id 1 --ext exp2 --text_path ./assets/text_prompt.txt
+```
+
+
+A few more parameters you may be interested:
+* `--repeat_times`: number of replications for generation, default `1`.
+* `--motion_length`: specify the number of poses for generation, only applicable in (a).
+
+The output files are stored under folder `./generation/<ext>/`. They are
+* `numpy files`: generated motions with shape of (nframe, 22, 3), under subfolder `./joints`.
+* `video files`: stick figure animation in mp4 format, under subfolder `./animation`.
+* `bvh files`: bvh files of the generated motion, under subfolder `./animation`.
+
+We also apply naive foot ik to the generated motions, see files with suffix `_ik`. It sometimes works well, but sometimes will fail.
+  
+</details>
+
+## :dancers: Visualization
+<details>
+
+All the animations are manually rendered in blender. We use the characters from [mixamo](https://www.mixamo.com/#/). You need to download the characters in T-Pose with skeleton.
+
+### Retargeting
+For retargeting, we found rokoko usually leads to large error on foot. On the other hand, [keemap.rig.transfer](https://github.com/nkeeline/Keemap-Blender-Rig-ReTargeting-Addon/releases) shows more precise retargetting. You could watch the [tutorial](https://www.youtube.com/watch?v=EG-VCMkVpxg) here.
+
+Following these steps:
+* Download keemap.rig.transfer from the github, and install it in blender.
+* Import both the motion files (.bvh) and character files (.fbx) in blender.
+* `Shift + Select` the both source and target skeleton. (Do not need to be Rest Position)
+* Switch to `Pose Mode`, then unfold the `KeeMapRig` tool at the top-right corner of the view window.
+* Load and read the bone mapping file `./assets/mapping.json`(or `mapping6.json` if it doesn't work). This file is manually made by us. It works for most characters in mixamo. You could make your own.
+* Adjust the `Number of Samples`, `Source Rig`, `Destination Rig Name`.
+* Clik `Transfer Animation from Source Destination`, wait a few seconds.
+
+We didn't tried other retargetting tools. Welcome to comment if you find others are more useful.
+
+### Scene
+
+We use this [scene](https://drive.google.com/file/d/1lg62nugD7RTAIz0Q_YP2iZsxpUzzOkT1/view?usp=sharing) for animation.
+
+
+</details>
+
+## :clapper: Temporal Inpainting
+<details>
+We conduct mask-based editing in the m-transformer stage, followed by the regeneration of residual tokens for the entire sequence. To load your own motion, provide the path through `--source_motion`. Utilize `-msec` to specify the mask section, supporting either ratio or frame index. For instance, `-msec 0.3,0.6` with `max_motion_length=196` is equivalent to `-msec 59,118`, indicating the editing of the frame section [59, 118]. 
+
+```
+python edit_t2m.py --gpu_id 1 --ext exp3 --use_res_model -msec 0.4,0.7 --text_prompt "A man picks something from the ground using his right hand."
+```
+
+Note: Presently, the source motion must adhere to the format of a HumanML3D dim-263 feature vector. An example motion vector data from the HumanML3D test set is available in `example_data/000612.npy`. To process your own motion data, you can utilize the `process_file` function from `utils/motion_process.py`.
+
+</details>
+
+## :space_invader: Train Your Own Models
+<details>
+
+
+**Note**: You have to train RVQ **BEFORE** training masked/residual transformers. The latter two can be trained simultaneously.
+
+### Train RVQ
+```
+python train_vq.py --name rvq_name --gpu_id 1 --dataset_name t2m --batch_size 512 --num_quantizers 6  --max_epoch 500 --quantize_drop_prob 0.2
+```
+
+### Train Masked Transformer
+```
+python train_t2m_transformer.py --name mtrans_name --gpu_id 2 --dataset_name t2m --batch_size 64 --vq_name rvq_name
+```
+
+### Train Residual Transformer
+```
+python train_res_transformer.py --name rtrans_name  --gpu_id 2 --dataset_name t2m --batch_size 64 --vq_name rvq_name --cond_drop_prob 0.2 --share_weight
+```
+
+* `--dataset_name`: motion dataset, `t2m` for HumanML3D and `kit` for KIT-ML.  
+* `--name`: name your model. This will create to model space as `./checkpoints/<dataset_name>/<name>`
+* `--gpu_id`: GPU id.
+* `--batch_size`: we use `512` for rvq training. For masked/residual transformer, we use `64` on HumanML3D and `16` for KIT-ML.
+* `--num_quantizers`: number of quantization layers, `6` is used in our case.
+* `--quantize_drop_prob`: quantization dropout ratio, `0.2` is used.
+* `--vq_name`: when training masked/residual transformer, you need to specify the name of rvq model for tokenization.
+* `--cond_drop_prob`: condition drop ratio, for classifier-free guidance. `0.2` is used.
+* `--share_weight`: whether to share the projection/embedding weights in residual transformer.
+
+All the pre-trained models and intermediate results will be saved in space `./checkpoints/<dataset_name>/<name>`.
+</details>
+
+## :book: Evaluation
+<details>
+
+### Evaluate RVQ Reconstruction:
+HumanML3D:
+```
+python eval_t2m_vq.py --gpu_id 0 --name rvq_nq6_dc512_nc512_noshare_qdp0.2 --dataset_name t2m --ext rvq_nq6
+
+```
+KIT-ML:
+```
+python eval_t2m_vq.py --gpu_id 0 --name rvq_nq6_dc512_nc512_noshare_qdp0.2_k --dataset_name kit --ext rvq_nq6
+```
+
+### Evaluate Text2motion Generation:
+HumanML3D:
+```
+python eval_t2m_trans_res.py --res_name tres_nlayer8_ld384_ff1024_rvq6ns_cdp0.2_sw --dataset_name t2m --name t2m_nlayer8_nhead6_ld384_ff1024_cdp0.1_rvq6ns --gpu_id 1 --cond_scale 4 --time_steps 10 --ext evaluation
+```
+KIT-ML:
+```
+python eval_t2m_trans_res.py --res_name tres_nlayer8_ld384_ff1024_rvq6ns_cdp0.2_sw_k --dataset_name kit --name t2m_nlayer8_nhead6_ld384_ff1024_cdp0.1_rvq6ns_k --gpu_id 0 --cond_scale 2 --time_steps 10 --ext evaluation
+```
+
+* `--res_name`: model name of `residual transformer`.  
+* `--name`: model name of `masked transformer`.  
+* `--cond_scale`: scale of classifer-free guidance.
+* `--time_steps`: number of iterations for inference.
+* `--ext`: filename for saving evaluation results.
+
+The final evaluation results will be saved in `./checkpoints/<dataset_name>/<name>/eval/<ext>.log`
+
+</details>
+
+## Acknowlegements
+
+We sincerely thank the open-sourcing of these works where our code is based on: 
+
+[deep-motion-editing](https://github.com/DeepMotionEditing/deep-motion-editing), [Muse](https://github.com/lucidrains/muse-maskgit-pytorch), [vector-quantize-pytorch](https://github.com/lucidrains/vector-quantize-pytorch), [T2M-GPT](https://github.com/Mael-zys/T2M-GPT), [MDM](https://github.com/GuyTevet/motion-diffusion-model/tree/main) and [MLD](https://github.com/ChenFengYe/motion-latent-diffusion/tree/main)
+
+## License
+This code is distributed under an [MIT LICENSE](https://github.com/EricGuo5513/momask-codes/tree/main?tab=MIT-1-ov-file#readme).
+
+Note that our code depends on other libraries, including SMPL, SMPL-X, PyTorch3D, and uses datasets which each have their own respective licenses that must also be followed.

app.py

@@ -0,0 +1,372 @@
+from functools import partial
+import os
+
+import torch
+import numpy as np
+import gradio as gr
+import random
+import shutil
+
+print(f"Is CUDA available: {torch.cuda.is_available()}")
+print(f"CUDA device: {torch.cuda.get_device_name(torch.cuda.current_device())}")
+
+import os
+from os.path import join as pjoin
+
+import torch.nn.functional as F
+
+from models.mask_transformer.transformer import MaskTransformer, ResidualTransformer
+from models.vq.model import RVQVAE, LengthEstimator
+
+from options.hgdemo_option import EvalT2MOptions
+from utils.get_opt import get_opt
+
+from utils.fixseed import fixseed
+from visualization.joints2bvh import Joint2BVHConvertor
+from torch.distributions.categorical import Categorical
+
+from utils.motion_process import recover_from_ric
+from utils.plot_script import plot_3d_motion
+
+from utils.paramUtil import t2m_kinematic_chain
+
+from gen_t2m import load_vq_model, load_res_model, load_trans_model, load_len_estimator
+
+clip_version = 'ViT-B/32'
+
+WEBSITE = """
+<div class="embed_hidden">
+<h1 style='text-align: center'> MoMask: Generative Masked Modeling of 3D Human Motions </h1>
+<h2 style='text-align: center'>
+<a href="https://ericguo5513.github.io" target="_blank"><nobr>Chuan Guo*</nobr></a> &emsp;
+<a href="https://yxmu.foo/" target="_blank"><nobr>Yuxuan Mu*</nobr></a> &emsp;
+<a href="https://scholar.google.com/citations?user=w4e-j9sAAAAJ&hl=en" target="_blank"><nobr>Muhammad Gohar Javed*</nobr></a> &emsp;
+<a href="https://sites.google.com/site/senwang1312home/" target="_blank"><nobr>Sen Wang</nobr></a> &emsp;
+<a href="https://www.ece.ualberta.ca/~lcheng5/" target="_blank"><nobr>Li Cheng</nobr></a>
+</h2>
+<h2 style='text-align: center'>
+<nobr>arXiv 2023</nobr>
+</h2>
+<h3 style="text-align:center;">
+<a target="_blank" href="https://arxiv.org/abs/2312.00063"> <button type="button" class="btn btn-primary btn-lg"> Paper </button></a> &ensp;
+<a target="_blank" href="https://github.com/EricGuo5513/momask-codes"> <button type="button" class="btn btn-primary btn-lg"> Code </button></a> &ensp;
+<a target="_blank" href="https://ericguo5513.github.io/momask/"> <button type="button" class="btn btn-primary btn-lg"> Webpage </button></a> &ensp;
+<a target="_blank" href="https://ericguo5513.github.io/source_files/momask_2023_bib.txt"> <button type="button" class="btn btn-primary btn-lg"> BibTex </button></a>
+</h3>
+<h3> Description </h3>
+<p>
+🔥🔥🔥 This space presents an interactive demo for <a href='https://ericguo5513.github.io/momask/' target='_blank'><b>MoMask</b></a>, a method for text-to-motion generation!!! It generates human motions (skeletal animations) based on your descriptions. To gain a better understanding of our work, you could try the provided examples first. 🔥🔥🔥
+</p>
+<p>
+🚀🚀🚀 In addition, we provide a link to download the generated human skeletal motion in <b>BVH</b> file format, compatible with CG software such as Blender!!! 🚀🚀🚀
+</p>
+<p>
+😁😁😁 If you find this demo interesting, we would appreciate your star on our <a href="https://github.com/EricGuo5513/momask-codes" target="_blank">github</a>. More details could be found on our <a href='https://ericguo5513.github.io/momask/' target='_blank'>webpage</a>. 🫶🫶🫶
+</p>
+<p>
+If you have any issues on this space or feature requests, we warmly welcome you to contact us through our <a href="https://github.com/EricGuo5513/momask-codes/issues" target="_blank">github repo</a> or <a href="mailto:ymu3@ualberta.ca?subject =[MoMask]Feedback&body = Message">email</a>.
+</p>
+</div>
+"""
+WEBSITE_bottom = """
+<p>
+We thanks <a href="https://huggingface.co/spaces/Mathux/TMR" target="_blank">TMR</a> for this cool space template.
+</p>
+</div>
+"""
+
+EXAMPLES = [
+   "A person is running on a treadmill.", "The person takes 4 steps backwards.", 
+   "A person jumps up and then lands.", "The person was pushed but did not fall.", 
+   "The person does a salsa dance.", "A figure streches it hands and arms above its head.",
+   "This person kicks with his right leg then jabs several times.",
+   "A person stands for few seconds and picks up his arms and shakes them.",
+   "A person walks in a clockwise circle and stops where he began.",
+   "A man bends down and picks something up with his right hand.",
+   "A person walks with a limp, their left leg gets injured.",
+   "A person repeatedly blocks their face with their right arm.",
+   "The person holds his left foot with his left hand, puts his right foot up and left hand up too.",
+   "A person stands, crosses left leg in front of the right, lowering themselves until they are sitting, both hands on the floor before standing and uncrossing legs.",
+   "The man walked forward, spun right on one foot and walked back to his original position.",
+   "A man is walking forward then steps over an object then continues walking forward.",
+]
+
+# Show closest text in the training
+
+
+# css to make videos look nice
+# var(--block-border-color); TODO
+CSS = """
+.generate_video {
+    position: relative;
+    margin-left: auto;
+    margin-right: auto;
+    box-shadow: var(--block-shadow);
+    border-width: var(--block-border-width);
+    border-color: #000000;
+    border-radius: var(--block-radius);
+    background: var(--block-background-fill);
+    width: 25%;
+    line-height: var(--line-sm);
+}
+}
+"""
+
+
+DEFAULT_TEXT = "A person is "
+
+
+if not os.path.exists("checkpoints/t2m"):
+    os.system("bash prepare/download_models_demo.sh")
+
+##########################
+######Preparing demo######
+##########################
+parser = EvalT2MOptions()
+opt = parser.parse()
+fixseed(opt.seed)
+opt.device = torch.device("cpu" if opt.gpu_id == -1 else "cuda:" + str(opt.gpu_id))
+dim_pose = 263
+root_dir = pjoin(opt.checkpoints_dir, opt.dataset_name, opt.name)
+model_dir = pjoin(root_dir, 'model')
+model_opt_path = pjoin(root_dir, 'opt.txt')
+model_opt = get_opt(model_opt_path, device=opt.device)
+
+######Loading RVQ######
+vq_opt_path = pjoin(opt.checkpoints_dir, opt.dataset_name, model_opt.vq_name, 'opt.txt')
+vq_opt = get_opt(vq_opt_path, device=opt.device)
+vq_opt.dim_pose = dim_pose
+vq_model, vq_opt = load_vq_model(vq_opt)
+
+model_opt.num_tokens = vq_opt.nb_code
+model_opt.num_quantizers = vq_opt.num_quantizers
+model_opt.code_dim = vq_opt.code_dim
+
+######Loading R-Transformer######
+res_opt_path = pjoin(opt.checkpoints_dir, opt.dataset_name, opt.res_name, 'opt.txt')
+res_opt = get_opt(res_opt_path, device=opt.device)
+res_model = load_res_model(res_opt, vq_opt, opt)
+
+assert res_opt.vq_name == model_opt.vq_name
+
+######Loading M-Transformer######
+t2m_transformer = load_trans_model(model_opt, opt, 'latest.tar')
+
+#####Loading Length Predictor#####
+length_estimator = load_len_estimator(model_opt)
+
+t2m_transformer.eval()
+vq_model.eval()
+res_model.eval()
+length_estimator.eval()
+
+res_model.to(opt.device)
+t2m_transformer.to(opt.device)
+vq_model.to(opt.device)
+length_estimator.to(opt.device)
+
+opt.nb_joints = 22
+mean = np.load(pjoin(opt.checkpoints_dir, opt.dataset_name, model_opt.vq_name, 'meta', 'mean.npy'))
+std = np.load(pjoin(opt.checkpoints_dir, opt.dataset_name, model_opt.vq_name, 'meta', 'std.npy'))
+def inv_transform(data):
+    return data * std + mean
+
+kinematic_chain = t2m_kinematic_chain
+converter = Joint2BVHConvertor()
+cached_dir = './cached'
+uid = 12138
+animation_path = pjoin(cached_dir, f'{uid}')
+os.makedirs(animation_path, exist_ok=True)
+
+@torch.no_grad()
+def generate(
+    text, uid, motion_length=0, use_ik=True, seed=10107, repeat_times=1,
+):
+    # fixseed(seed)
+    print(text)
+    prompt_list = []
+    length_list = []
+    est_length = False
+    prompt_list.append(text)
+    if motion_length == 0:
+        est_length = True
+    else:
+        length_list.append(motion_length)
+
+    if est_length:
+        print("Since no motion length are specified, we will use estimated motion lengthes!!")
+        text_embedding = t2m_transformer.encode_text(prompt_list)
+        pred_dis = length_estimator(text_embedding)
+        probs = F.softmax(pred_dis, dim=-1)  # (b, ntoken)
+        token_lens = Categorical(probs).sample()  # (b, seqlen)
+    else:
+        token_lens = torch.LongTensor(length_list) // 4
+        token_lens = token_lens.to(opt.device).long()
+
+    m_length = token_lens * 4
+    captions = prompt_list
+    datas = []
+    for r in range(repeat_times):
+        mids = t2m_transformer.generate(captions, token_lens,
+                                        timesteps=opt.time_steps,
+                                        cond_scale=opt.cond_scale,
+                                        temperature=opt.temperature,
+                                        topk_filter_thres=opt.topkr,
+                                        gsample=opt.gumbel_sample)
+        mids = res_model.generate(mids, captions, token_lens, temperature=1, cond_scale=5)
+        pred_motions = vq_model.forward_decoder(mids)
+        pred_motions = pred_motions.detach().cpu().numpy()
+        data = inv_transform(pred_motions)
+        ruid = random.randrange(99999)
+        for k, (caption, joint_data)  in enumerate(zip(captions, data)):
+            animation_path = pjoin(cached_dir, f'{uid}')
+            os.makedirs(animation_path, exist_ok=True)
+            joint_data = joint_data[:m_length[k]]
+            joint = recover_from_ric(torch.from_numpy(joint_data).float(), 22).numpy()
+            bvh_path = pjoin(animation_path, "sample_repeat%d.bvh" % (r))
+            save_path = pjoin(animation_path, "sample_repeat%d_%d.mp4"%(r, ruid))
+            if use_ik:
+                print("Using IK")
+                _, joint = converter.convert(joint, filename=bvh_path, iterations=100)
+            else:
+                _, joint = converter.convert(joint, filename=bvh_path, iterations=100, foot_ik=False)
+            plot_3d_motion(save_path, kinematic_chain, joint, title=caption, fps=20)
+            np.save(pjoin(animation_path, "sample_repeat%d.npy"%(r)), joint)
+        data_unit = {
+            "url": pjoin(animation_path, "sample_repeat%d_%d.mp4"%(r, ruid))
+            }
+        datas.append(data_unit)
+
+    return datas
+
+
+# HTML component
+def get_video_html(data, video_id, width=700, height=700):
+    url = data["url"]
+    # class="wrap default svelte-gjihhp hide"
+    # <div class="contour_video" style="position: absolute; padding: 10px;">
+    # width="{width}" height="{height}"
+    video_html = f"""
+<h2 style='text-align: center'>
+<a href="file/{pjoin(animation_path, "sample_repeat0.bvh")}" download="sample.bvh"><b>BVH Download</b></a>
+</h2>
+<video class="generate_video" width="{width}" height="{height}" style="center" preload="auto" muted playsinline onpause="this.load()"
+autoplay loop disablepictureinpicture id="{video_id}">
+  <source src="file/{url}" type="video/mp4">
+  Your browser does not support the video tag.
+</video>
+"""
+    return video_html
+
+def generate_component(generate_function, text, motion_len='0', postprocess='IK'):
+    if text == DEFAULT_TEXT or text == "" or text is None:
+        return [None for _ in range(1)]
+    # uid = random.randrange(99999)
+    try:
+        motion_len = max(0, min(int(float(motion_len) * 20), 196))
+    except:
+        motion_len = 0
+    use_ik = postprocess == 'IK'
+    datas = generate_function(text, uid, motion_len, use_ik)
+    htmls = [get_video_html(data, idx) for idx, data in enumerate(datas)]
+    return htmls
+
+
+# LOADING
+
+# DEMO
+theme = gr.themes.Default(primary_hue="blue", secondary_hue="gray")
+generate_and_show = partial(generate_component, generate)
+
+with gr.Blocks(css=CSS, theme=theme) as demo:
+    gr.Markdown(WEBSITE)
+    videos = []
+
+    with gr.Row():
+        with gr.Column(scale=3):
+            text = gr.Textbox(
+                show_label=True,
+                label="Text prompt",
+                value=DEFAULT_TEXT,
+            )
+            with gr.Row():
+                with gr.Column(scale=1):
+                    motion_len = gr.Textbox(
+                        show_label=True,
+                        label="Motion length (<10s)",
+                        value=0,
+                        info="Specify the motion length; 0 to use the default auto-setting.",
+                    )
+                with gr.Column(scale=1):
+                    use_ik = gr.Radio(
+                        ["Raw", "IK"],
+                        label="Post-processing",
+                        value="IK",
+                        info="Use basic inverse kinematic (IK) for foot contact locking",
+                    )
+            gen_btn = gr.Button("Generate", variant="primary")
+            clear = gr.Button("Clear", variant="secondary")
+            gr.Markdown(
+                        f"""
+                            
+                        """
+                    )
+
+        with gr.Column(scale=2):
+
+            def generate_example(text):
+                return generate_and_show(text)
+
+            examples = gr.Examples(
+                examples=[[x, None, None] for x in EXAMPLES],
+                inputs=[text],
+                examples_per_page=10,
+                run_on_click=False,
+                cache_examples=False,
+                fn=generate_example,
+                outputs=[],
+            )
+
+    i = -1
+    # should indent
+    for _ in range(1):
+        with gr.Row():
+            for _ in range(1):
+                i += 1
+                video = gr.HTML()
+                videos.append(video)
+    gr.Markdown(WEBSITE_bottom)
+    # connect the examples to the output
+    # a bit hacky
+    examples.outputs = videos
+
+    def load_example(example_id):
+        processed_example = examples.non_none_processed_examples[example_id]
+        return gr.utils.resolve_singleton(processed_example)
+
+    examples.dataset.click(
+        load_example,
+        inputs=[examples.dataset],
+        outputs=examples.inputs_with_examples,  # type: ignore
+        show_progress=False,
+        postprocess=False,
+        queue=False,
+    ).then(fn=generate_example, inputs=examples.inputs, outputs=videos)
+
+    gen_btn.click(
+        fn=generate_and_show,
+        inputs=[text, motion_len, use_ik],
+        outputs=videos,
+    )
+    text.submit(
+        fn=generate_and_show,
+        inputs=[text, motion_len, use_ik],
+        outputs=videos,
+    )
+
+    def clear_videos():
+        return [None for x in range(1)] + [DEFAULT_TEXT]
+
+    clear.click(fn=clear_videos, outputs=videos + [text])
+
+demo.launch(share=True)

assets/mapping.json

@@ -0,0 +1 @@
+{"bones": [{"name": "Hips", "label": "", "description": "", "SourceBoneName": "Hips", "DestinationBoneName": "mixamorig:Hips", "keyframe_this_bone": true, "CorrectionFactorX": 2.6179938316345215, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": true, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 0.2588190734386444, "QuatCorrectionFactorx": 0.965925931930542, "QuatCorrectionFactory": 2.7939677238464355e-09, "QuatCorrectionFactorz": -2.7939677238464355e-09, "scale_secondary_bone_name": ""}, {"name": "RightUpLeg", "label": "", "description": "", "SourceBoneName": "RightUpLeg", "DestinationBoneName": "mixamorig:RightUpLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftUpLeg", "label": "", "description": "", "SourceBoneName": "LeftUpLeg", "DestinationBoneName": "mixamorig:LeftUpLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightLeg", "label": "", "description": "", "SourceBoneName": "RightLeg", "DestinationBoneName": "mixamorig:RightLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 2.094395160675049, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftLeg", "label": "", "description": "", "SourceBoneName": "LeftLeg", "DestinationBoneName": "mixamorig:LeftLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 3.665191411972046, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightShoulder", "label": "", "description": "", "SourceBoneName": "RightShoulder", "DestinationBoneName": "mixamorig:RightShoulder", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftShoulder", "label": "", "description": "", "SourceBoneName": "LeftShoulder", "DestinationBoneName": "mixamorig:LeftShoulder", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightArm", "label": "", "description": "", "SourceBoneName": "RightArm", "DestinationBoneName": "mixamorig:RightArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": -1.0471975803375244, "CorrectionFactorZ": -0.1745329201221466, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftArm", "label": "", "description": "", "SourceBoneName": "LeftArm", "DestinationBoneName": "mixamorig:LeftArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 1.0471975803375244, "CorrectionFactorZ": 0.1745329201221466, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightForeArm", "label": "", "description": "", "SourceBoneName": "RightForeArm", "DestinationBoneName": "mixamorig:RightForeArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": -2.094395160675049, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftForeArm", "label": "", "description": "", "SourceBoneName": "LeftForeArm", "DestinationBoneName": "mixamorig:LeftForeArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 1.5707963705062866, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Spine", "label": "", "description": "", "SourceBoneName": "Spine", "DestinationBoneName": "mixamorig:Spine", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Spine1", "label": "", "description": "", "SourceBoneName": "Spine1", "DestinationBoneName": "mixamorig:Spine1", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Spine2", "label": "", "description": "", "SourceBoneName": "Spine2", "DestinationBoneName": "mixamorig:Spine2", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Neck", "label": "", "description": "", "SourceBoneName": "Neck", "DestinationBoneName": "mixamorig:Neck", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Head", "label": "", "description": "", "SourceBoneName": "Head", "DestinationBoneName": "mixamorig:Head", "keyframe_this_bone": true, "CorrectionFactorX": 0.3490658402442932, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightFoot", "label": "", "description": "", "SourceBoneName": "RightFoot", "DestinationBoneName": "mixamorig:RightFoot", "keyframe_this_bone": true, "CorrectionFactorX": -0.19192171096801758, "CorrectionFactorY": 2.979980945587158, "CorrectionFactorZ": -0.05134282633662224, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": -0.082771435379982, "QuatCorrectionFactorx": -0.0177358016371727, "QuatCorrectionFactory": -0.9920229315757751, "QuatCorrectionFactorz": -0.09340716898441315, "scale_secondary_bone_name": ""}, {"name": "LeftFoot", "label": "", "description": "", "SourceBoneName": "LeftFoot", "DestinationBoneName": "mixamorig:LeftFoot", "keyframe_this_bone": true, "CorrectionFactorX": -0.25592508912086487, "CorrectionFactorY": -2.936899423599243, "CorrectionFactorZ": 0.2450830191373825, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 0.11609010398387909, "QuatCorrectionFactorx": 0.10766097158193588, "QuatCorrectionFactory": -0.9808290004730225, "QuatCorrectionFactorz": -0.11360746622085571, "scale_secondary_bone_name": ""}], "start_frame_to_apply": 0, "number_of_frames_to_apply": 196, "keyframe_every_n_frames": 1, "source_rig_name": "bvh_batch1_sample30_repeat1_len48", "destination_rig_name": "Armature", "bone_rotation_mode": "EULER", "bone_mapping_file": "C:\\Users\\cguo2\\Documents\\CVPR2024_MoMask\\mapping.json"}

assets/mapping6.json

@@ -0,0 +1 @@
+{"bones": [{"name": "Hips", "label": "", "description": "", "SourceBoneName": "Hips", "DestinationBoneName": "mixamorig6:Hips", "keyframe_this_bone": true, "CorrectionFactorX": 2.6179938316345215, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": true, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 0.2588190734386444, "QuatCorrectionFactorx": 0.965925931930542, "QuatCorrectionFactory": 2.7939677238464355e-09, "QuatCorrectionFactorz": -2.7939677238464355e-09, "scale_secondary_bone_name": ""}, {"name": "RightUpLeg", "label": "", "description": "", "SourceBoneName": "RightUpLeg", "DestinationBoneName": "mixamorig6:RightUpLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftUpLeg", "label": "", "description": "", "SourceBoneName": "LeftUpLeg", "DestinationBoneName": "mixamorig6:LeftUpLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightLeg", "label": "", "description": "", "SourceBoneName": "RightLeg", "DestinationBoneName": "mixamorig6:RightLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 2.094395160675049, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftLeg", "label": "", "description": "", "SourceBoneName": "LeftLeg", "DestinationBoneName": "mixamorig6:LeftLeg", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 3.665191411972046, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightShoulder", "label": "", "description": "", "SourceBoneName": "RightShoulder", "DestinationBoneName": "mixamorig6:RightShoulder", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftShoulder", "label": "", "description": "", "SourceBoneName": "LeftShoulder", "DestinationBoneName": "mixamorig6:LeftShoulder", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightArm", "label": "", "description": "", "SourceBoneName": "RightArm", "DestinationBoneName": "mixamorig6:RightArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": -1.0471975803375244, "CorrectionFactorZ": -0.1745329201221466, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftArm", "label": "", "description": "", "SourceBoneName": "LeftArm", "DestinationBoneName": "mixamorig6:LeftArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 1.0471975803375244, "CorrectionFactorZ": 0.1745329201221466, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "RightForeArm", "label": "", "description": "", "SourceBoneName": "RightForeArm", "DestinationBoneName": "mixamorig6:RightForeArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": -2.094395160675049, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "LeftForeArm", "label": "", "description": "", "SourceBoneName": "LeftForeArm", "DestinationBoneName": "mixamorig6:LeftForeArm", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 1.5707963705062866, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Spine", "label": "", "description": "", "SourceBoneName": "Spine", "DestinationBoneName": "mixamorig6:Spine", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Spine1", "label": "", "description": "", "SourceBoneName": "Spine1", "DestinationBoneName": "mixamorig6:Spine1", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Spine2", "label": "", "description": "", "SourceBoneName": "Spine2", "DestinationBoneName": "mixamorig6:Spine2", "keyframe_this_bone": true, "CorrectionFactorX": 0.0, "CorrectionFactorY": 0.0, "CorrectionFactorZ": 0.0, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 1.0, "QuatCorrectionFactorx": 0.0, "QuatCorrectionFactory": 0.0, "QuatCorrectionFactorz": 0.0, "scale_secondary_bone_name": ""}, {"name": "Neck", "label": "", "description": "", "SourceBoneName": "Neck", "DestinationBoneName": "mixamorig6:Neck", "keyframe_this_bone": true, "CorrectionFactorX": -0.994345486164093, "CorrectionFactorY": -0.006703000050038099, "CorrectionFactorZ": 0.04061730206012726, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 0.8787809014320374, "QuatCorrectionFactorx": -0.4767816960811615, "QuatCorrectionFactory": -0.01263047568500042, "QuatCorrectionFactorz": 0.016250507906079292, "scale_secondary_bone_name": ""}, {"name": "Head", "label": "", "description": "", "SourceBoneName": "Head", "DestinationBoneName": "mixamorig6:Head", "keyframe_this_bone": true, "CorrectionFactorX": -0.07639937847852707, "CorrectionFactorY": 0.011205507442355156, "CorrectionFactorZ": 0.011367863975465298, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": 0.9992374181747437, "QuatCorrectionFactorx": -0.038221005350351334, "QuatCorrectionFactory": 0.0053814793936908245, "QuatCorrectionFactorz": 0.005893632769584656, "scale_secondary_bone_name": ""}, {"name": "RightFoot", "label": "", "description": "", "SourceBoneName": "RightFoot", "DestinationBoneName": "mixamorig6:RightFoot", "keyframe_this_bone": true, "CorrectionFactorX": -0.17194896936416626, "CorrectionFactorY": 2.7372374534606934, "CorrectionFactorZ": -0.029542576521635056, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": -0.20128199458122253, "QuatCorrectionFactorx": 0.002824343740940094, "QuatCorrectionFactory": -0.9761614799499512, "QuatCorrectionFactorz": -0.08115538209676743, "scale_secondary_bone_name": ""}, {"name": "LeftFoot", "label": "", "description": "", "SourceBoneName": "LeftFoot", "DestinationBoneName": "mixamorig6:LeftFoot", "keyframe_this_bone": true, "CorrectionFactorX": -0.09363158047199249, "CorrectionFactorY": -2.9336421489715576, "CorrectionFactorZ": -0.17343592643737793, "has_twist_bone": false, "TwistBoneName": "", "set_bone_position": false, "set_bone_rotation": true, "bone_rotation_application_axis": "XYZ", "position_correction_factorX": 0.0, "position_correction_factorY": 0.0, "position_correction_factorZ": 0.0, "position_gain": 1.0, "position_pole_distance": 0.30000001192092896, "postion_type": "SINGLE_BONE_OFFSET", "set_bone_scale": false, "scale_gain": 1.0, "scale_max": 1.0, "scale_min": 0.5, "bone_scale_application_axis": "Y", "QuatCorrectionFactorw": -0.09925344586372375, "QuatCorrectionFactorx": 0.09088610112667084, "QuatCorrectionFactory": 0.9893556833267212, "QuatCorrectionFactorz": 0.05535021424293518, "scale_secondary_bone_name": ""}], "start_frame_to_apply": 0, "number_of_frames_to_apply": 196, "keyframe_every_n_frames": 1, "source_rig_name": "MoMask__02_ik", "destination_rig_name": "Armature", "bone_rotation_mode": "EULER", "bone_mapping_file": "C:\\Users\\cguo2\\Documents\\CVPR2024_MoMask\\mapping6.json"}

assets/text_prompt.txt

@@ -0,0 +1,12 @@
+the person holds his left foot with his left hand, puts his right foot up and left hand up too.#132
+a man bends down and picks something up with his left hand.#84
+A man stands for few seconds and picks up his arms and shakes them.#176
+A person walks with a limp, their left leg get injured.#192
+a person jumps up and then lands.#52
+a person performs a standing back kick.#52
+A person pokes their right hand along the ground, like they might be planting seeds.#60
+the person steps forward and uses the left leg to kick something forward.#92
+the man walked forward, spun right on one foot and walked back to his original position.#92
+the person was pushed but did not fall.#124
+this person stumbles left and right while moving forward.#132
+a person reaching down and picking something up.#148

checkpoints/kit/Comp_v6_KLD005/meta/mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e23fac51db2215ab5666324226be48f27efd6a6e7b22ebd17c28e0f056a7c22
+size 2136

checkpoints/kit/Comp_v6_KLD005/meta/std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:296a60656cea07e65ee64512d73d47c0412df0698b35194116330661be32fa90
+size 2136

checkpoints/kit/Comp_v6_KLD005/opt.txt

@@ -0,0 +1,54 @@
+------------ Options -------------
+batch_size: 32
+checkpoints_dir: ./checkpoints
+dataset_name: kit
+decomp_name: Decomp_SP001_SM001_H512
+dim_att_vec: 512
+dim_dec_hidden: 1024
+dim_movement2_dec_hidden: 512
+dim_movement_dec_hidden: 512
+dim_movement_enc_hidden: 512
+dim_movement_latent: 512
+dim_msd_hidden: 512
+dim_pos_hidden: 1024
+dim_pri_hidden: 1024
+dim_seq_de_hidden: 512
+dim_seq_en_hidden: 512
+dim_text_hidden: 512
+dim_z: 128
+early_stop_count: 3
+estimator_mod: bigru
+eval_every_e: 5
+feat_bias: 5
+fixed_steps: 5
+gpu_id: 2
+input_z: False
+is_continue: True
+is_train: True
+lambda_fake: 10
+lambda_gan_l: 0.1
+lambda_gan_mt: 0.1
+lambda_gan_mv: 0.1
+lambda_kld: 0.005
+lambda_rec: 1
+lambda_rec_init: 1
+lambda_rec_mot: 1
+lambda_rec_mov: 1
+log_every: 50
+lr: 0.0002
+max_sub_epoch: 50
+max_text_len: 20
+n_layers_dec: 1
+n_layers_msd: 2
+n_layers_pos: 1
+n_layers_pri: 1
+n_layers_seq_de: 2
+n_layers_seq_en: 1
+name: Comp_v6_KLD005
+num_experts: 4
+save_every_e: 10
+save_latest: 500
+text_enc_mod: bigru
+tf_ratio: 0.4
+unit_length: 4
+-------------- End ----------------

checkpoints/kit/text_mot_match/eval/E005.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.134 2.099 3.881 3.921 2.285 3.744 2.655 1.721 1.946 2.250 2.924 6.876 3.502 1.861 5.818 3.507 8.488 1.692 1.776 4.583 1.980 2.727 3.955 3.607 5.916 3.186 3.832 1.700 1.664 2.896 3.318 2.128
+Negative Pairs Distance
+4.801 7.064 6.329 6.437 3.465 6.363 8.897 6.655 10.889 6.358 8.022 8.131 3.472 9.457 10.489 3.636 9.595 9.930 12.090 6.514 10.054 2.828 11.206 9.073 6.163 10.645 7.251 7.684 13.491 3.869 8.233 5.459

checkpoints/kit/text_mot_match/eval/E010.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.908 2.165 2.165 3.126 1.697 1.459 2.442 1.858 1.139 1.622 2.096 2.478 4.929 3.777 4.985 2.417 1.533 2.688 4.114 2.632 3.061 2.188 5.152 2.518 3.600 4.669 8.488 2.839 3.971 2.343 3.527 3.871
+Negative Pairs Distance
+11.756 2.332 11.629 8.415 1.482 5.168 9.379 10.146 11.044 11.025 11.965 5.885 10.289 5.902 8.258 11.976 8.073 7.024 9.437 4.496 3.879 7.686 4.651 4.576 2.901 14.143 2.428 8.259 5.828 7.087 12.836 13.889

checkpoints/kit/text_mot_match/eval/E015.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.850 4.731 5.109 2.265 1.610 2.926 2.168 2.637 2.084 1.510 2.175 2.338 1.857 5.098 1.801 2.207 2.303 1.409 2.606 1.605 2.917 1.752 3.372 2.717 3.066 2.843 2.329 2.640 2.341 4.156 3.331 8.131
+Negative Pairs Distance
+6.084 8.782 9.299 5.081 10.574 12.375 6.629 3.773 12.802 17.637 15.346 12.001 11.877 8.885 5.704 7.574 10.524 11.066 13.124 9.736 2.497 16.383 10.255 14.209 15.128 12.731 10.557 15.367 11.058 11.331 2.683 9.325

checkpoints/kit/text_mot_match/eval/E020.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.720 2.590 1.074 3.083 2.166 2.835 6.362 2.816 2.871 1.793 3.421 6.261 1.957 5.514 2.695 2.254 1.668 1.689 4.970 2.880 1.581 4.509 2.255 3.921 2.240 2.384 2.844 2.736 4.322 3.335 3.728 2.677
+Negative Pairs Distance
+11.181 11.564 13.729 7.366 12.419 12.882 8.641 18.567 7.485 7.284 11.086 8.577 5.952 4.970 14.443 13.611 11.813 10.937 13.638 11.140 14.285 8.947 13.830 14.733 11.218 3.280 2.429 11.807 11.222 9.967 10.158 10.779

checkpoints/kit/text_mot_match/eval/E025.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.553 1.793 2.145 1.160 1.568 0.923 2.591 7.707 2.747 12.233 5.216 1.305 1.507 2.035 1.736 2.155 1.751 1.618 2.041 2.239 2.825 2.352 3.072 2.020 1.761 5.428 2.252 1.604 2.319 2.266 3.278 3.328
+Negative Pairs Distance
+11.462 3.411 1.404 11.581 3.764 11.409 14.073 4.259 12.997 3.549 7.593 12.030 12.991 7.647 12.337 12.592 14.496 14.496 13.530 13.371 12.775 14.828 11.200 17.627 9.128 9.617 9.297 6.782 14.615 2.304 9.306 10.946

checkpoints/kit/text_mot_match/eval/E030.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+7.378 4.702 10.228 2.154 7.611 0.943 1.770 1.255 5.552 3.161 2.180 2.252 3.278 1.767 1.787 1.450 1.546 2.051 1.476 1.828 1.829 1.423 1.150 2.198 9.395 1.748 2.700 2.304 3.917 4.163 2.811 2.562
+Negative Pairs Distance
+3.884 11.919 5.835 6.898 5.753 19.333 10.732 16.049 10.197 11.546 13.682 12.106 12.993 11.981 13.373 13.464 11.788 11.267 16.719 9.747 4.581 14.785 13.978 9.505 7.465 11.392 11.481 9.757 11.077 16.272 15.701 12.256

checkpoints/kit/text_mot_match/eval/E035.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+2.095 9.020 2.176 1.030 1.653 1.306 1.963 4.513 1.906 1.186 2.731 1.678 1.395 2.232 1.394 4.946 1.057 4.450 3.671 2.931 3.655 1.374 1.361 6.001 5.699 1.579 10.237 2.940 1.622 2.032 7.194 1.069
+Negative Pairs Distance
+13.345 14.638 14.061 19.142 8.843 12.152 15.600 7.368 15.635 13.863 15.644 3.686 13.388 15.807 20.432 11.571 13.876 8.513 13.236 19.330 8.771 14.880 13.005 10.810 12.430 10.102 4.440 18.083 3.135 13.201 5.465 8.700

checkpoints/kit/text_mot_match/eval/E040.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+2.531 1.118 4.384 1.787 1.595 2.184 1.371 1.264 2.086 1.477 1.123 1.240 1.040 11.428 3.469 1.511 1.782 2.528 6.645 1.197 1.967 1.188 3.484 5.019 1.024 1.892 3.185 1.017 2.682 1.116 1.047 1.691
+Negative Pairs Distance
+12.010 18.700 16.704 10.839 15.398 8.070 9.340 14.570 8.946 18.806 4.114 12.699 11.821 4.733 15.875 14.064 17.230 16.277 14.135 22.007 18.454 13.210 8.825 8.937 9.880 16.306 13.555 13.649 5.738 12.752 12.994 1.158

checkpoints/kit/text_mot_match/eval/E045.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.412 2.614 2.471 0.842 2.629 7.116 7.275 5.793 1.034 8.645 4.851 2.357 1.155 1.208 1.168 1.698 1.550 1.132 7.423 2.531 1.147 2.240 1.575 1.232 1.455 2.134 1.421 3.723 2.887 1.759 3.806 4.387
+Negative Pairs Distance
+10.150 13.459 14.645 13.875 8.746 12.398 13.303 16.393 22.565 9.265 13.325 9.629 18.867 15.043 18.596 19.413 17.903 8.695 4.948 13.620 14.171 19.699 15.496 11.753 12.914 17.017 7.990 14.401 11.768 8.648 13.926 13.402

checkpoints/kit/text_mot_match/eval/E050.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+5.304 2.162 1.767 7.675 1.446 5.696 2.277 0.717 1.477 12.431 3.673 1.286 2.633 1.883 2.555 1.399 11.572 1.303 3.411 1.521 3.885 0.984 1.210 1.038 5.024 5.886 1.283 1.026 2.056 1.738 3.654 2.659
+Negative Pairs Distance
+17.692 17.993 15.431 10.788 22.819 16.591 23.417 13.203 7.453 17.521 19.102 17.595 7.883 20.475 11.371 17.178 5.380 4.884 17.968 18.298 13.822 19.112 11.842 10.838 14.131 14.906 16.252 14.499 13.878 2.300 11.417 21.558

checkpoints/kit/text_mot_match/eval/E055.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.286 11.980 3.768 1.701 1.978 1.173 1.752 5.425 4.705 13.314 1.451 1.055 2.002 1.749 1.538 5.364 2.637 1.023 1.074 3.722 2.823 2.851 3.961 11.326 1.716 1.712 3.175 2.536 2.563 1.154 1.269 3.001
+Negative Pairs Distance
+11.300 3.898 15.139 13.965 1.849 22.030 13.080 15.480 10.960 11.965 4.839 9.320 3.329 24.126 10.234 8.670 18.491 14.462 14.877 7.870 10.906 13.899 6.250 5.214 8.553 16.769 11.940 18.139 23.584 9.895 10.384 5.684

checkpoints/kit/text_mot_match/eval/E060.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+2.646 12.210 0.830 2.921 0.875 1.999 4.337 0.802 2.268 0.840 2.059 2.004 0.628 2.383 4.972 0.918 2.044 4.344 1.900 2.817 3.564 2.183 1.713 1.272 1.134 1.917 5.561 2.390 2.127 3.047 3.518 2.000
+Negative Pairs Distance
+6.734 13.725 24.744 24.194 14.176 12.985 4.384 8.463 14.464 20.511 13.927 15.659 18.104 10.338 14.063 18.091 20.923 11.184 18.401 24.048 9.389 11.433 10.597 22.686 12.311 3.344 17.366 14.234 20.743 16.628 18.863 6.987

checkpoints/kit/text_mot_match/eval/E065.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+4.282 9.482 1.553 1.383 1.122 2.035 0.854 1.958 1.230 1.450 2.783 5.676 0.971 1.129 1.624 1.782 2.330 7.208 1.222 4.682 1.120 2.564 3.288 1.271 1.158 0.989 1.165 1.431 1.322 3.795 1.797 3.976
+Negative Pairs Distance
+10.757 12.124 8.528 18.803 11.622 10.307 18.852 9.454 13.769 24.235 15.302 4.133 9.539 19.628 16.231 15.288 18.951 18.389 14.687 9.998 8.974 20.162 12.884 14.608 19.159 19.726 16.222 4.654 16.910 15.950 4.430 9.514

checkpoints/kit/text_mot_match/eval/E070.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.853 1.746 2.313 1.558 7.835 0.945 1.523 7.543 0.974 1.104 2.812 1.116 2.340 1.406 1.499 3.407 1.224 2.075 5.423 2.600 1.321 6.505 1.368 8.134 3.274 2.626 2.342 3.057 4.097 2.022 1.990 1.068
+Negative Pairs Distance
+16.888 5.090 20.032 15.900 10.658 9.950 24.814 10.751 8.165 16.934 14.184 18.338 13.667 6.515 12.078 7.889 9.116 26.239 10.249 16.732 15.455 16.731 7.098 9.890 5.493 7.409 13.144 13.131 9.515 1.967 7.075 12.000

checkpoints/kit/text_mot_match/eval/E075.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+5.996 5.991 6.329 11.539 1.356 2.672 1.162 1.664 7.148 1.469 1.583 4.772 1.021 2.245 1.263 2.734 1.628 0.797 2.095 2.180 6.602 1.719 4.648 1.462 1.192 6.518 0.926 7.409 1.480 1.979 2.509 3.288
+Negative Pairs Distance
+10.215 10.800 19.266 6.959 6.636 25.179 16.741 19.729 11.336 16.523 16.567 10.568 15.567 13.981 4.875 8.074 1.630 8.896 20.647 15.145 17.009 20.560 13.781 23.201 18.929 18.235 7.325 26.645 19.469 20.427 15.561 15.197

checkpoints/kit/text_mot_match/eval/E080.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.944 1.884 4.347 1.715 4.907 1.845 1.217 9.310 1.924 2.145 1.969 1.843 3.612 0.980 1.484 5.777 2.220 1.096 1.829 1.210 1.392 4.746 2.195 2.041 6.412 0.769 5.934 12.347 1.315 1.303 5.924 1.596
+Negative Pairs Distance
+20.166 23.190 19.774 17.923 13.037 21.200 22.463 14.820 14.639 14.424 20.394 15.858 17.843 19.721 9.291 13.911 6.487 18.521 16.954 15.071 21.124 3.136 21.915 26.672 10.117 19.504 16.569 15.500 19.505 14.488 12.333 11.741

checkpoints/kit/text_mot_match/eval/E085.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.160 1.123 0.681 0.707 2.277 2.063 2.552 1.243 3.954 3.408 1.125 1.530 3.594 1.932 1.488 1.558 1.368 1.740 4.621 0.844 1.054 1.013 1.795 0.552 2.498 0.922 2.624 0.899 1.455 1.543 1.219 2.387
+Negative Pairs Distance
+12.292 15.500 11.375 14.608 23.654 3.728 2.467 13.880 16.046 21.566 21.143 17.623 4.482 17.418 22.232 17.313 26.219 22.449 13.927 21.166 10.767 5.275 1.208 22.217 2.804 21.347 16.782 18.786 19.475 19.334 14.207 16.465

checkpoints/kit/text_mot_match/eval/E090.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.012 2.578 5.767 1.255 0.855 2.173 4.103 1.101 13.313 17.991 1.063 1.962 0.998 2.235 3.338 1.534 4.680 0.950 1.804 1.357 1.058 0.933 1.001 5.634 3.647 1.667 2.159 12.595 2.218 3.775 2.603 1.347
+Negative Pairs Distance
+1.622 24.847 4.103 23.529 12.966 16.220 3.390 10.709 17.506 5.613 9.319 21.534 20.392 18.943 14.061 8.722 18.443 16.522 19.102 12.086 23.014 25.803 15.898 6.899 19.726 23.200 19.056 8.438 20.097 16.868 14.529 12.901

checkpoints/kit/text_mot_match/eval/E095.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.957 5.130 1.610 0.692 0.973 2.350 1.095 0.812 10.008 2.255 1.878 2.190 4.425 1.911 1.336 4.439 0.943 10.657 1.138 1.613 2.040 5.536 1.493 0.938 0.889 1.510 8.307 3.408 2.917 3.500 2.125 2.211
+Negative Pairs Distance
+13.302 14.787 22.667 23.635 20.254 23.571 8.536 14.727 18.053 16.699 12.599 14.799 15.496 9.799 9.621 16.289 1.345 13.394 17.885 21.358 23.233 18.082 17.676 13.628 11.943 12.769 29.724 28.740 23.942 21.485 16.927 12.710

checkpoints/kit/text_mot_match/eval/E100.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+15.756 1.123 4.602 0.836 2.995 0.605 1.258 4.576 5.112 0.662 0.605 2.008 1.062 1.816 0.632 9.981 1.660 0.973 0.711 2.293 1.124 5.743 1.202 4.518 1.711 4.652 0.785 0.745 0.526 3.104 1.319 4.840
+Negative Pairs Distance
+10.425 22.640 4.875 19.405 11.291 16.246 17.498 23.780 22.589 10.619 8.373 9.351 16.751 14.980 22.690 22.484 12.499 24.838 24.494 23.126 14.764 12.877 10.966 11.744 22.366 10.996 14.653 5.500 18.956 24.887 14.791 15.646

checkpoints/kit/text_mot_match/eval/E105.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.089 2.387 6.936 1.425 1.880 2.587 1.990 1.701 1.069 0.980 1.417 9.058 1.353 1.204 2.475 2.691 0.932 2.726 1.296 1.698 6.379 1.387 15.032 1.641 3.618 1.276 1.370 3.261 0.990 7.931 2.744 3.607
+Negative Pairs Distance
+18.003 20.362 11.659 15.585 13.766 14.232 21.723 4.696 8.918 19.056 13.097 24.289 12.855 17.362 22.556 10.135 14.734 14.375 14.859 16.949 9.689 14.875 15.761 10.648 13.664 18.514 11.027 11.611 18.166 5.020 17.382 17.951

checkpoints/kit/text_mot_match/eval/E110.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+4.594 16.486 2.173 5.006 0.736 5.348 2.516 1.424 1.905 1.518 4.290 0.630 1.854 0.662 6.736 1.189 3.855 0.870 3.669 0.790 0.969 2.029 2.629 1.290 1.532 7.682 6.683 0.917 4.618 6.050 3.025 0.723
+Negative Pairs Distance
+12.923 22.179 24.706 19.398 9.272 7.455 13.993 6.839 13.685 16.117 14.054 19.742 5.135 15.211 10.941 18.224 11.478 16.176 10.114 18.356 20.260 16.683 27.581 13.941 8.018 7.124 12.105 17.994 0.854 15.453 12.923 15.096

checkpoints/kit/text_mot_match/eval/E115.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+4.260 5.487 7.693 4.676 1.321 1.041 1.544 1.485 9.702 1.870 2.287 3.421 4.685 3.229 8.235 4.715 2.157 2.155 11.826 1.385 13.426 1.473 6.342 1.925 6.983 2.693 0.856 1.752 5.750 2.104 2.385 4.791
+Negative Pairs Distance
+20.920 17.386 17.106 6.395 14.940 9.904 15.653 8.603 12.403 25.547 25.790 4.830 15.455 18.065 3.405 14.249 26.497 8.641 10.089 12.739 22.130 18.371 10.461 19.548 15.514 18.363 10.790 9.796 13.923 19.527 2.634 12.634

checkpoints/kit/text_mot_match/eval/E120.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.613 10.684 3.614 1.907 1.610 1.017 3.122 1.261 3.162 0.933 1.599 2.377 0.762 1.865 2.864 1.124 1.034 3.213 2.520 1.569 0.995 0.759 4.635 5.838 0.870 1.485 1.530 2.547 8.226 2.399 1.487 3.931
+Negative Pairs Distance
+18.579 23.097 22.131 12.888 30.161 26.140 12.161 25.980 13.960 24.155 2.520 6.033 26.685 7.331 8.465 16.310 24.265 19.109 19.149 16.481 14.715 17.881 20.127 13.153 4.347 20.907 20.906 8.501 15.853 23.314 15.491 25.455

checkpoints/kit/text_mot_match/model/finest.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f786d733ca1bfa5dc1474554f167e1f608ec629868061e51ccfdb812ea61e6a
+size 245481907

checkpoints/t2m/Comp_v6_KLD005/meta/mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bdb5ba69a3a9e34d71990db15bc535ebc024c8d95ddb5574196f96058faa7d3
+size 2232

checkpoints/t2m/Comp_v6_KLD005/meta/std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a5f7d60301c9465972fc225f8ad0ee8f957e7720431189123eb6d15873a9557
+size 2232

checkpoints/t2m/Comp_v6_KLD005/opt.txt

@@ -0,0 +1,54 @@
+------------ Options -------------
+batch_size: 32
+checkpoints_dir: ./checkpoints
+dataset_name: t2m
+decomp_name: Decomp_SP001_SM001_H512
+dim_att_vec: 512
+dim_dec_hidden: 1024
+dim_movement2_dec_hidden: 512
+dim_movement_dec_hidden: 512
+dim_movement_enc_hidden: 512
+dim_movement_latent: 512
+dim_msd_hidden: 512
+dim_pos_hidden: 1024
+dim_pri_hidden: 1024
+dim_seq_de_hidden: 512
+dim_seq_en_hidden: 512
+dim_text_hidden: 512
+dim_z: 128
+early_stop_count: 3
+estimator_mod: bigru
+eval_every_e: 5
+feat_bias: 5
+fixed_steps: 5
+gpu_id: 1
+input_z: False
+is_continue: False
+is_train: True
+lambda_fake: 10
+lambda_gan_l: 0.1
+lambda_gan_mt: 0.1
+lambda_gan_mv: 0.1
+lambda_kld: 0.005
+lambda_rec: 1
+lambda_rec_init: 1
+lambda_rec_mot: 1
+lambda_rec_mov: 1
+log_every: 50
+lr: 0.0002
+max_sub_epoch: 50
+max_text_len: 20
+n_layers_dec: 1
+n_layers_msd: 2
+n_layers_pos: 1
+n_layers_pri: 1
+n_layers_seq_de: 2
+n_layers_seq_en: 1
+name: Comp_v6_KLD005
+num_experts: 4
+save_every_e: 10
+save_latest: 500
+text_enc_mod: bigru
+tf_ratio: 0.4
+unit_length: 4
+-------------- End ----------------

checkpoints/t2m/rvq_name/opt.txt

@@ -0,0 +1,44 @@
+------------ Options -------------
+batch_size: 256
+checkpoints_dir: ./checkpoints
+code_dim: 512
+commit: 0.02
+dataset_name: t2m
+depth: 3
+dilation_growth_rate: 3
+do_vq_res: False
+down_t: 2
+eval_every_e: 1
+ext: default
+feat_bias: 5
+gamma: 0.1
+gpu_id: 0
+is_continue: False
+is_train: True
+log_every: 10
+loss_vel: 0.5
+lr: 0.0002
+max_epoch: 50
+milestones: [150000, 250000]
+mu: 0.99
+n_res: 2
+name: rvq_name
+nb_code: 512
+num_quantizers: 6
+output_emb_width: 512
+quantize_dropout_prob: 0.2
+recons_loss: l1_smooth
+save_every_e: 2
+save_latest: 500
+seed: 3407
+shared_codebook: False
+stride_t: 2
+vq_act: relu
+vq_name: rvq_nq6_dc512_nc512_noshare_qdp0.2
+vq_norm: None
+warm_up_iter: 2000
+weight_decay: 0.0
+which_epoch: all
+width: 512
+window_size: 64
+-------------- End ----------------

checkpoints/t2m/text_mot_match/eval/E005.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+2.138 3.585 6.431 3.316 2.915 3.651 2.846 3.037 6.336 3.522 3.202 2.891 1.960 5.782 1.933 4.873 3.097 3.693 3.603 3.285 2.163 7.284 3.414 1.798 3.902 4.261 5.328 3.147 6.786 7.201 3.807 4.556 1.279 6.056 4.641 1.930 3.137 3.403 4.191 4.825 3.522 1.962 1.947 8.036 3.557 4.207 3.009 2.873 3.439 1.936 9.244 2.556 3.661 2.426 3.827 2.633 4.978 3.241 1.883 4.369 1.605 6.167 3.187 2.980
+Negative Pairs Distance
+7.684 7.251 7.515 13.336 11.317 8.791 11.320 7.274 7.530 10.009 10.813 8.484 8.580 8.130 6.765 8.351 8.458 4.622 10.156 8.999 7.133 5.604 6.739 6.342 4.268 10.279 3.411 8.958 5.971 11.234 6.465 9.877 6.681 3.099 7.299 3.563 9.506 7.835 4.392 5.835 5.587 11.703 7.469 5.336 11.653 9.488 9.204 10.227 8.903 11.845 8.152 11.568 5.678 11.725 11.399 7.016 7.917 7.193 11.800 10.289 8.340 6.820 14.050 7.414

checkpoints/t2m/text_mot_match/eval/E010.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+2.368 0.934 3.339 2.851 7.548 4.274 4.616 3.087 5.109 3.182 2.390 3.640 1.201 2.870 2.700 4.310 2.962 1.834 2.689 1.684 5.466 4.030 1.856 2.129 2.856 2.080 1.374 1.521 1.930 4.614 4.484 1.493 2.287 3.146 3.296 3.164 1.294 6.759 1.899 1.356 1.339 1.435 2.758 1.028 2.244 3.313 1.439 1.873 5.779 6.083 6.621 2.469 3.199 1.684 1.338 4.343 1.805 1.842 2.437 1.985 2.152 1.773 1.916 2.397
+Negative Pairs Distance
+3.371 6.222 12.795 11.448 11.061 5.772 10.886 9.773 9.096 3.671 7.492 9.996 10.521 10.957 11.268 5.102 12.331 8.901 13.735 13.542 6.837 9.248 12.835 9.278 7.703 10.815 14.478 5.701 7.244 8.938 3.909 13.006 8.206 12.380 7.833 10.313 11.688 9.678 15.328 10.780 6.664 5.745 3.885 11.306 8.481 3.590 3.059 10.034 7.081 7.480 10.011 2.261 4.195 12.070 6.496 1.780 10.264 8.738 12.774 12.661 3.845 2.913 5.139 10.449

checkpoints/t2m/text_mot_match/eval/E015.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+5.635 4.365 1.758 7.120 2.122 5.028 3.057 1.999 1.407 1.492 1.576 3.012 2.568 1.865 3.120 4.290 6.581 1.971 1.502 3.113 4.361 1.652 2.058 7.970 5.700 6.587 1.295 2.932 2.793 4.732 4.453 1.880 5.972 2.598 1.801 2.391 2.672 2.691 2.551 0.991 4.353 2.636 2.225 4.888 5.664 9.172 0.939 0.823 0.978 0.918 2.677 2.839 0.878 5.426 5.099 1.188 2.365 3.993 2.732 5.112 4.739 1.685 3.245 2.932
+Negative Pairs Distance
+8.349 8.690 8.668 8.618 11.899 5.328 9.142 12.406 11.802 6.766 6.286 6.416 13.772 5.873 5.219 4.214 7.341 11.516 8.086 8.696 8.727 12.731 10.625 9.442 11.745 10.770 8.947 9.977 10.015 5.765 11.373 12.010 7.621 4.839 13.131 11.073 11.632 10.954 2.635 12.538 8.563 2.470 6.850 6.131 8.921 11.868 9.324 7.270 3.399 3.792 11.799 6.229 14.861 10.649 4.161 10.082 4.159 9.676 8.362 10.114 14.197 12.460 2.946 11.656

checkpoints/t2m/text_mot_match/eval/E020.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.639 2.593 4.484 3.229 1.908 3.809 5.975 1.788 2.816 1.236 2.470 4.534 4.969 4.274 1.663 3.368 2.780 7.354 3.718 4.035 6.723 3.376 2.353 3.740 1.692 3.493 1.461 0.775 1.864 1.333 2.718 2.153 2.152 2.403 4.073 1.636 3.009 4.557 1.290 3.028 6.190 2.665 1.714 2.362 1.464 1.493 1.685 2.305 3.953 3.443 2.440 2.157 7.674 1.337 3.640 0.910 1.613 3.918 1.287 12.478 3.788 2.357 1.609 1.419
+Negative Pairs Distance
+7.379 12.135 11.961 8.214 7.093 15.171 5.768 7.391 6.999 5.967 5.915 9.818 15.420 5.949 5.837 9.039 6.797 5.483 9.227 1.807 10.707 8.089 12.895 5.841 9.760 8.882 12.048 11.364 2.663 11.144 12.087 11.463 8.490 7.476 6.095 8.174 14.009 13.326 11.961 10.205 3.341 6.644 9.152 12.635 12.135 11.879 9.290 8.208 10.543 4.041 13.544 15.427 9.436 10.737 13.831 12.030 7.694 4.468 15.355 2.236 4.427 3.901 12.986 14.113

checkpoints/t2m/text_mot_match/eval/E025.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+2.160 3.123 2.673 1.274 6.288 3.064 2.156 1.779 1.664 2.188 3.982 5.247 3.170 2.361 4.161 2.367 1.652 2.336 3.467 5.782 5.258 1.314 3.722 5.492 1.663 5.732 2.062 2.351 2.587 1.571 2.409 2.411 6.146 5.193 4.207 3.013 1.431 3.187 1.100 1.344 1.625 3.711 0.917 6.620 4.080 2.230 1.884 1.805 2.046 1.792 1.178 1.756 2.029 3.268 1.328 1.794 1.493 6.562 2.734 2.635 2.326 4.999 1.729 7.867
+Negative Pairs Distance
+5.004 5.192 7.421 12.019 11.470 9.982 6.755 9.042 9.402 4.459 10.686 8.709 14.472 10.902 11.827 11.461 9.566 11.826 13.496 11.107 9.148 10.478 13.818 6.955 10.105 5.290 11.824 13.777 6.761 11.292 17.203 10.521 10.090 13.692 11.677 10.680 10.203 6.210 12.732 9.553 12.763 11.885 4.563 12.447 6.661 6.581 3.495 8.131 10.747 10.149 11.988 11.166 2.840 12.352 14.597 7.480 8.204 10.761 2.073 8.824 14.124 9.904 8.432 10.644

checkpoints/t2m/text_mot_match/eval/E030.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+5.769 2.701 5.603 3.461 3.501 2.584 5.095 2.065 1.358 2.446 4.603 2.641 2.895 1.471 2.466 1.607 0.930 1.819 1.353 3.113 7.465 4.344 3.375 4.831 1.779 2.750 2.875 2.941 2.247 0.988 1.050 2.338 3.626 1.762 0.796 2.131 1.694 1.386 2.158 3.976 2.969 1.641 2.860 1.886 1.497 1.490 1.896 3.927 1.520 1.449 1.085 3.901 3.171 4.814 2.366 2.735 5.815 1.122 1.129 4.953 1.609 5.023 2.228 2.773
+Negative Pairs Distance
+11.844 13.121 7.596 12.851 13.889 11.492 9.487 2.997 13.484 7.261 1.788 9.130 9.110 7.852 8.295 9.320 11.202 11.790 13.054 9.690 12.577 3.168 6.269 10.308 11.983 5.900 10.662 11.556 13.187 13.133 12.790 9.736 7.152 12.138 10.264 13.381 10.001 11.219 9.328 10.502 12.704 7.180 10.973 9.989 10.436 4.979 14.441 5.024 11.842 11.142 16.077 6.655 13.333 8.766 14.357 13.773 7.105 11.169 9.949 9.969 12.078 12.049 9.408 13.646

checkpoints/t2m/text_mot_match/eval/E035.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+3.015 2.759 7.591 6.053 3.381 2.494 4.691 1.533 2.759 2.886 4.554 3.377 1.076 1.264 2.097 1.899 10.654 3.292 5.365 1.063 1.628 5.412 6.907 2.985 7.071 3.881 1.433 4.854 1.252 2.352 5.969 5.683 5.698 1.381 1.272 1.471 2.439 3.102 1.047 4.069 0.912 0.682 4.318 7.923 1.435 2.243 1.504 4.846 1.574 1.434 1.413 2.317 4.098 1.012 7.257 0.815 1.268 1.275 1.205 6.220 1.553 3.097 1.121 4.317
+Negative Pairs Distance
+8.111 10.381 8.601 8.611 10.391 14.334 8.258 12.430 1.033 9.069 7.270 15.955 9.880 6.052 9.510 8.462 11.041 16.363 12.415 10.365 14.795 9.277 11.011 9.625 5.031 11.133 7.795 5.386 13.971 9.357 11.929 6.353 6.024 10.256 11.845 13.513 6.557 6.254 13.107 9.092 13.515 6.420 1.810 12.100 5.244 12.693 13.093 6.583 12.862 12.863 8.406 15.206 7.722 12.308 11.860 15.117 12.917 9.394 14.406 6.435 12.583 10.266 9.806 7.909

checkpoints/t2m/text_mot_match/eval/E040.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.091 3.158 7.683 7.239 1.099 3.226 2.940 3.941 2.575 3.817 3.114 9.059 3.588 8.397 4.210 0.761 3.761 1.051 2.982 1.111 2.080 2.381 0.834 2.018 0.626 0.741 2.794 2.219 2.869 1.454 1.465 2.574 3.113 2.554 1.089 2.221 1.143 3.417 7.127 2.091 1.884 7.085 2.724 4.812 1.123 1.671 2.098 1.180 1.987 2.836 2.942 1.101 1.353 1.907 4.502 1.756 1.712 3.689 3.915 3.732 1.178 4.434 1.426 4.765
+Negative Pairs Distance
+7.758 7.423 12.026 11.045 4.614 18.689 9.143 7.832 7.377 5.228 5.019 13.617 9.827 6.091 9.724 5.717 4.499 10.508 15.079 12.134 8.691 10.813 18.536 12.898 14.130 12.709 6.524 14.031 8.108 8.034 18.086 14.097 6.817 13.653 4.394 15.235 13.638 12.487 12.820 14.682 11.744 11.734 15.113 12.353 15.548 7.648 11.888 8.054 4.929 10.516 5.412 15.665 2.656 9.560 6.485 8.409 15.143 14.369 11.108 12.582 8.295 13.409 8.260 15.920

checkpoints/t2m/text_mot_match/eval/E045.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.229 5.692 8.703 1.241 3.887 1.769 2.470 8.832 1.336 1.730 1.015 2.745 2.991 3.538 2.202 2.632 4.694 6.508 2.805 2.564 1.648 1.522 4.542 3.489 5.952 3.880 0.938 7.360 6.155 3.456 4.902 0.875 1.802 1.712 2.043 0.982 9.952 2.507 6.213 1.753 2.948 1.832 1.512 1.218 1.036 7.176 9.888 3.051 1.263 2.009 0.868 4.171 3.859 2.438 8.382 7.390 5.269 6.246 3.971 3.855 5.653 4.777 1.835 4.219
+Negative Pairs Distance
+7.079 3.908 9.954 15.469 17.535 17.224 11.547 10.721 14.723 13.506 11.699 13.040 13.515 15.012 10.038 12.103 11.218 9.423 9.647 15.224 15.121 9.937 11.791 13.358 11.099 13.063 11.833 8.263 13.041 13.143 6.902 10.864 10.572 16.783 14.272 7.396 12.496 12.448 7.637 15.296 14.801 8.080 11.159 12.617 9.477 10.468 5.623 12.924 12.892 15.975 11.561 12.399 15.255 10.662 10.685 8.176 12.925 9.704 5.554 14.692 9.740 16.954 15.803 4.117

checkpoints/t2m/text_mot_match/eval/E050.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.988 8.534 3.090 1.506 0.546 3.149 2.591 1.457 2.084 2.224 2.506 7.658 1.488 5.345 8.596 3.019 1.280 1.224 3.789 2.969 4.153 3.922 3.468 2.677 0.549 2.438 2.397 1.058 0.694 4.246 2.182 9.738 3.974 4.391 3.852 0.978 1.228 1.394 1.914 5.302 2.232 2.375 1.958 0.804 3.267 4.355 1.425 1.856 2.107 2.300 1.773 1.005 1.508 2.442 2.032 1.787 5.650 1.220 1.837 0.949 0.894 2.533 1.709 2.203
+Negative Pairs Distance
+10.846 1.982 10.140 14.580 8.048 10.215 8.266 11.341 15.567 14.373 12.174 9.238 10.762 11.636 9.484 17.984 15.269 10.753 12.488 14.978 12.295 16.706 5.277 12.151 17.022 12.385 11.328 11.967 2.335 7.675 11.572 10.201 4.607 12.996 4.525 10.899 7.531 18.636 10.677 6.494 5.820 15.886 13.994 6.347 9.459 12.818 8.411 10.870 11.731 11.338 13.900 10.221 11.980 1.934 15.422 14.012 10.747 13.300 9.517 8.904 13.915 1.789 2.982 1.413

checkpoints/t2m/text_mot_match/eval/E055.txt

@@ -0,0 +1,4 @@
+Positive Pairs Distance
+1.247 3.241 3.085 5.559 3.278 3.504 1.281 5.672 3.510 2.796 2.979 4.566 1.734 1.347 6.664 1.833 1.697 3.458 2.551 9.985 10.803 6.760 4.255 2.079 1.631 3.465 3.195 1.208 7.328 2.997 3.596 3.009 2.351 0.688 2.198 0.714 5.610 3.208 4.415 2.061 7.265 3.046 1.876 6.715 1.057 1.153 1.743 1.395 2.131 9.693 1.005 1.567 2.103 6.292 2.028 7.331 5.032 4.028 1.292 0.818 1.527 1.678 2.556 0.867
+Negative Pairs Distance
+14.511 9.498 16.638 16.673 10.296 8.035 17.646 12.668 7.180 14.810 9.057 14.211 15.495 19.012 16.264 7.727 9.027 14.481 16.910 10.368 11.140 12.842 10.332 12.354 12.907 7.463 6.222 12.132 10.834 9.261 9.624 13.718 12.051 7.325 15.345 9.044 8.045 12.457 18.816 16.320 8.970 10.849 10.921 8.175 14.154 10.171 15.705 13.980 12.632 9.271 10.276 11.752 2.374 14.594 5.849 13.037 14.157 15.233 6.680 15.071 5.267 12.505 11.125 3.867