axolotl-ai-cloud/axolotl

Please check that this issue hasn't been reported before. I searched previous Bug Reports didn't find any similar reports. Expected Behavior shoud run clean Current behaviour With FSDP2, cpu_ram_efficient_loading: true and fsdp_config.activation_checkpointing: true together crash when the full state dict is broadcast, before training starts. The model doesn't matter: it reproduces on a tiny Qwen3-MoE and on GLM-5.3-Flash. KeyError: "state dict entry 'base_model.model.model.layers.0.self_attn.q_proj.base_layer.weight' is neither a parameter nor a buffer" File "src/axolotl/monkeypatch/accelerate/fsdp2.py", line 385, in fsdp2_load_full_state_dict File "src/axolotl/monkeypatch/accelerate/fsdp2.py", line 57, in _state_dict_entry Steps to reproduce axolotl train repro.yaml --num-processes 2 Config yaml base_model: ./tiny-qwen3moe datasets: - path: fozziethebeat/alpaca_messages_2k_test type: chat_template val_set_size: 0.05 output_dir: ./out adapter: lora sequence_len: 2048 samp

⚠️ Please check that this feature request hasn't been suggested before. I searched previous Ideas in Discussions didn't find any similar feature requests. I searched previous Issues didn't find any similar feature requests. 🔖 Feature description axolotl export (#3950) only converts a full / merged checkpoint via convert_hf_to_gguf.py. llama.cpp also ships convert_lora_to_gguf.py, which turns a PEFT adapter directory into a standalone GGUF LoRA that runtimes load on top of a base GGUF: llama-cli -m base-f16.gguf --lora my-adapter.gguf ✔️ Solution Dispatch: in axolotl.cli.export.resolve_model_dir, when output_dir holds an adapter (adapter_config.json present, no merged/), call convert_lora_to_gguf.py <adapter_dir> --base --outfile … --outtype … instead of raising. The script reads base_model_name_or_path from adapter_config.json (which axolotl already writes); pass cfg.base_model as --base when it is a local path so no Hub fetch is needed. Config: auto-detect from th

Description Choosing the right LoRA rank is largely trial-and-error. An automatic recommendation based on dataset size, task type, and base model would save time. Use Case New users don't know what rank to choose Different tasks benefit from different ranks (classification vs generation) Could be a --auto-rank flag or a preprocessing analysis step Based on our experience, rank 16 works for most tasks under 10K samples, but this heuristic could be formalized.

⚠️ Please check that this feature request hasn't been suggested before. I searched previous Ideas in Discussions didn't find any similar feature requests. I searched previous Issues didn't find any similar feature requests. 🔖 Feature description Hey team, Creating a separate issue for this thread. I was wondering if there were any plans for including TPU/Trainium support in the near future? Axolotl is currently on accelerate v1.13.0, which supports both platforms. I have linked relevant documentation below Trainium support was just added in 1.13.0. TPU support has existed for longer, but isn't particularly well documented. https://huggingface.co/docs/accelerate/basic_tutorials/tpu https://huggingface.co/docs/accelerate/concept_guides/training_tpu @NanoCode012 mentioned potential issues such as difficulty including the platform in dev/CI. Would be great to hear what the team thinks and if TPU would be feasible! ✔️ Solution Support the same, or a useful subset of, the fe