by SakanaAI
Hypernetworks that update LLMs to remember factual information
# Add to your Claude Code skills
git clone https://github.com/SakanaAI/doc-to-loraLast scanned: 5/8/2026
{
"issues": [],
"status": "PASSED",
"scannedAt": "2026-05-08T05:57:52.334Z",
"semgrepRan": false,
"npmAuditRan": true,
"pipAuditRan": true
}doc-to-lora is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by SakanaAI. Hypernetworks that update LLMs to remember factual information. It has 805 GitHub stars.
Yes. doc-to-lora passed SkillsLLM's automated security scan — a dependency vulnerability audit plus prompt-injection heuristics — with no high-severity issues. You can read the full report in the Security Report section on this page.
Clone the repository with "git clone https://github.com/SakanaAI/doc-to-lora" and add it to your Claude Code skills directory (see the Installation section above).
doc-to-lora is primarily written in Python. It is open-source under SakanaAI on GitHub, so you can review or fork the full source.
Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh doc-to-lora against similar tools.
No comments yet. Be the first to share your thoughts!
⚠️ Third-Party Software Notice
This skill is third-party open-source software developed and hosted independently on GitHub. SkillsLLM is an informational directory and does not control or maintain the underlying repository.
Any security checks, ratings, or warnings displayed by SkillsLLM are automated and limited in scope. They do not constitute a security certification or guarantee that the software is safe, error-free, or free from malicious code, vulnerabilities, compromised dependencies, or prompt-injection risks.
Review the source code, permissions, dependencies, and configuration before installing or running any third-party skill. Use is at your own risk. To the maximum extent permitted by applicable law, SkillsLLM is not liable for losses arising from third-party software.
curl -LsSf https://astral.sh/uv/install.sh | sh
./install.sh
uv run huggingface-cli login
uv run huggingface-cli download SakanaAI/doc-to-lora --local-dir trained_d2l --include "*/"
# caveat: this interface only supports non-batched inputs
# for batched inference please see `src/ctx_to_lora/modeling/hypernet.py`
import torch
from ctx_to_lora.model_loading import get_tokenizer
from ctx_to_lora.modeling.hypernet import ModulatedPretrainedModel
# model loading
checkpoint_path = "trained_d2l/gemma_demo/checkpoint-80000/pytorch_model.bin"
state_dict = torch.load(checkpoint_path, weights_only=False)
model = ModulatedPretrainedModel.from_state_dict(
state_dict, train=False, use_sequence_packing=False
)
model.reset()
tokenizer = get_tokenizer(model.base_model.name_or_path)
# prepare data
doc = open("data/sakana_wiki.txt", "r").read()
chat = [{"role": "user", "content": "Tell me about Sakana AI."}]
chat_ids = tokenizer.apply_chat_template(
chat,
add_special_tokens=False,
return_attention_mask=False,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
# calls after internalization will be influenced by internalized info
model.internalize(doc)
outputs = model.generate(input_ids=chat_ids, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))
# remove internalized info
# model.reset()
# without internalized info, the model will halucinate
# outputs = model.generate(input_ids=chat_ids, max_new_tokens=512)
# print(tokenizer.decode(outputs[0]))
uv run demo/app.py
To run any of the following scripts, use uv run $PATH_TO_SCRIPT from the root of this project.
| Experiment | Data prep | Training | Evaluation | Notes |
|---|---|---|---|---|
| Main experiment | scripts/main_exp/0-download_data.sh |
scripts/main_exp/1-train.sh |
scripts/main_exp/eval/*.sh |
Downloading data is fastest; regenerate only if you need fresh synthetic data. Evaluation scripts reproduce the main paper metrics. |
| NIAH | scripts/niah/0-gen_data.sh |
scripts/niah/1-train.sh |
scripts/niah/2-eval.sh |
Run the scripts in order; data generation only needs to happen once |
After downloading/generating the data, we can see samples of the data using this script.
uv run webui/self_gen_viewer.py
See more info at webui/SELF_GEN_VIEWER.md.
@inproceedings{charakorn2026doctolora,
title ={Doc-to-Lo{RA}: Learning to Instantly Internalize Contexts},
author ={Rujikorn Charakorn and Edoardo Cetin and Shinnosuke Uesaka and Robert Tjarko Lange},
booktitle ={Forty-third International Conference on Machine Learning},
year ={2026},
url ={https://openreview.net/forum?id=iW1oBBO72S}
}