by SakanaAI
Hypernetworks that update LLMs to remember factual information
# Add to your Claude Code skills
git clone https://github.com/SakanaAI/doc-to-loraLast scanned: 5/8/2026
{
"issues": [],
"status": "PASSED",
"scannedAt": "2026-05-08T05:57:52.334Z",
"semgrepRan": false,
"npmAuditRan": true,
"pipAuditRan": true
}doc-to-lora is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by SakanaAI. Hypernetworks that update LLMs to remember factual information. It has 779 GitHub stars.
Yes. doc-to-lora passed SkillsLLM's automated security scan — a dependency vulnerability audit plus prompt-injection heuristics — with no high-severity issues. You can read the full report in the Security Report section on this page.
Clone the repository with "git clone https://github.com/SakanaAI/doc-to-lora" and add it to your Claude Code skills directory (see the Installation section above).
doc-to-lora is primarily written in Python. It is open-source under SakanaAI on GitHub, so you can review or fork the full source.
Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh doc-to-lora against similar tools.
No comments yet. Be the first to share your thoughts!
curl -LsSf https://astral.sh/uv/install.sh | sh
./install.sh
uv run huggingface-cli login
uv run huggingface-cli download SakanaAI/doc-to-lora --local-dir trained_d2l --include "*/"
# caveat: this interface only supports non-batched inputs
# for batched inference please see `src/ctx_to_lora/modeling/hypernet.py`
import torch
from ctx_to_lora.model_loading import get_tokenizer
from ctx_to_lora.modeling.hypernet import ModulatedPretrainedModel
# model loading
checkpoint_path = "trained_d2l/gemma_demo/checkpoint-80000/pytorch_model.bin"
state_dict = torch.load(checkpoint_path, weights_only=False)
model = ModulatedPretrainedModel.from_state_dict(
state_dict, train=False, use_sequence_packing=False
)
model.reset()
tokenizer = get_tokenizer(model.base_model.name_or_path)
# prepare data
doc = open("data/sakana_wiki.txt", "r").read()
chat = [{"role": "user", "content": "Tell me about Sakana AI."}]
chat_ids = tokenizer.apply_chat_template(
chat,
add_special_tokens=False,
return_attention_mask=False,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
# calls after internalization will be influenced by internalized info
model.internalize(doc)
outputs = model.generate(input_ids=chat_ids, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))
# remove internalized info
# model.reset()
# without internalized info, the model will halucinate
# outputs = model.generate(input_ids=chat_ids, max_new_tokens=512)
# print(tokenizer.decode(outputs[0]))
uv run demo/app.py
To run any of the following scripts, use uv run $PATH_TO_SCRIPT from the root of this project.
| Experiment | Data prep | Training | Evaluation | Notes |
|---|---|---|---|---|
| Main experiment | scripts/main_exp/0-download_data.sh |
scripts/main_exp/1-train.sh |
scripts/main_exp/eval/*.sh |
Downloading data is fastest; regenerate only if you need fresh synthetic data. Evaluation scripts reproduce the main paper metrics. |
| NIAH | scripts/niah/0-gen_data.sh |
scripts/niah/1-train.sh |
scripts/niah/2-eval.sh |
Run the scripts in order; data generation only needs to happen once |
After downloading/generating the data, we can see samples of the data using this script.
uv run webui/self_gen_viewer.py
See more info at webui/SELF_GEN_VIEWER.md.
@inproceedings{charakorn2026doctolora,
title ={Doc-to-Lo{RA}: Learning to Instantly Internalize Contexts},
author ={Rujikorn Charakorn and Edoardo Cetin and Shinnosuke Uesaka and Robert Tjarko Lange},
booktitle ={Forty-third International Conference on Machine Learning},
year ={2026},
url ={https://openreview.net/forum?id=iW1oBBO72S}
}