---
            library_name: custom
            tags:
            - robotics
            - diffusion
            - mixture-of-experts
            - multi-modal
            license: mit
            datasets:
            - CALVIN
            language:
            - en
            pipeline_tag: robotics
            ---
            # MoDE (Mixture 1of Diffusion Experts) Model

            This model implements a Mixture of Diffusion Experts architecture for robotic manipulation, combining transformer-based processing with expert routing and diffusion-based action prediction.

            ## Model Architecture
            - Base Architecture: MoDE with custom Mixture of Experts Transformer
            - Vision Encoder: {getattr(model_instance, 'resnet_type', 'ResNet')} with FiLM conditioning
            - EMA: Enabled
            - Action Window Size: {model_instance.act_window_size}
            - Sampling Steps: {model_instance.num_sampling_steps}
            - Sampler Type: {model_instance.sampler_type}

            ## Input/Output Specifications
            - RGB Static Camera: (B, T, 3, H, W) tensor
            - RGB Gripper Camera: (B, T, 3, H, W) tensor
            - Language Instructions: Text strings
            - Output: (B, T, 7) tensor representing 7-DoF actions

            ## Usage Example
            ```python
            from huggingface_hub import hf_hub_download
            import torch

            weights_path = hf_hub_download(repo_id="{repo_name}", filename="model_cleaned.safetensors")
            model.load_pretrained_parameters(weights_path)

            obs = {
                "rgb_obs": {
                    "rgb_static": static_image,
                    "rgb_gripper": gripper_image
                }
            }
            goal = {"lang_text": "pick up the blue cube"}
            action = model.step(obs, goal)
            ```

            ## Training Configuration
            - Optimizer: AdamW
            - Learning Rate: {config.optimizer.learning_rate}
            - Weight Decay: {config.optimizer.transformer_weight_decay}