 Andrej
|
1ccbaf4416
|
nit delete redundant catch/raise in execute
Remove redundant exception handling in chdir
|
2025-10-29 08:10:03 -07:00 |
|
|
Andrej
|
29ff38d94b
|
Merge pull request #35 from bhaskar0210s/master
fix: return inf instead of crashing when evaluate_bpb has zero total_bytes
|
2025-10-29 08:06:24 -07:00 |
|
|
Andrej
|
a1de1f46ad
|
Merge pull request #156 from tlepoint/fix/export-base-dir
Export the base dir variable in runcpu.sh
|
2025-10-28 15:19:08 -07:00 |
|
|
Andrej
|
ee00f523d0
|
fixing all the typos to make the pull requests stop
Batch of typo fixes
|
2025-10-28 13:36:07 -07:00 |
|
|
Andrej Karpathy
|
fdda5826e3
|
Merge branch 'haowei01-fix_kv_cache_due_to_resize'
|
2025-10-28 16:54:30 +00:00 |
|
|
Andrej Karpathy
|
baf0b3fdda
|
also add a test that failed before the fix and passes now with the fix for kv cache resize
|
2025-10-28 16:54:17 +00:00 |
|
|
Andrej Karpathy
|
f1db6b4712
|
delete czar call for help, i'm working through the inbound on that now. add current LLM policy which just asks for disclosure atm
|
2025-10-28 16:51:41 +00:00 |
|
|
Andrej Karpathy
|
9415931f85
|
delete czar call for help, i'm working through the inbound on that now. add current LLM policy which just asks for disclosure atm
|
2025-10-28 15:17:43 +00:00 |
|
|
Andrej Karpathy
|
c75fe54aa7
|
readme tweak, link to new discussion and add file structure
|
2025-10-25 19:39:16 +00:00 |
|
|
Andrej Karpathy
|
05a051dbe9
|
fix tokenization bug, there should be no space before first letter. sigh
|
2025-10-24 15:06:06 +00:00 |
|
|
Andrej Karpathy
|
8892470f29
|
add the SpellingBee task so that nanochat can count r in strawberry etc. along the way we had to add a bunch of new functionality, e.g. extend the calculator to support the count function of python. possibly the current TaskMixture uses way too many synthetic examples of SpellingBee because the eval gives us exactly 100% performance on spelling. We can tune this later to reclaim some wall clock time here I think
|
2025-10-24 14:02:48 +00:00 |
|
|
Andrej Karpathy
|
81597cd616
|
move the lr schedule args up in base_train so they are tunable in configurator
|
2025-10-24 13:27:31 +00:00 |
|
|
Andrej Karpathy
|
cc3636b01c
|
allow the tokenizer visualize_tokenization to also print the exact token id. you can never be paranoid enough
|
2025-10-24 13:27:05 +00:00 |
|
|
Andrej Karpathy
|
5eeb2b6ef9
|
experiment: looking to 'hire' a nanochat repo czar to help the repo, mentioning in readme
|
2025-10-22 16:55:54 +00:00 |
|
|
Andrej Karpathy
|
2dda5c4c8d
|
Merge branch 'ulanch-fix/ios-safari-input-overlap'
|
2025-10-22 16:26:35 +00:00 |
|
|
Andrej Karpathy
|
80b203ea59
|
also bump run1000.sh to new uv sync
|
2025-10-22 16:25:36 +00:00 |
|
|
Andrej Karpathy
|
51e70f0d3c
|
Merge branch 'lukestanley-fix-cpu-support-with-extras'
|
2025-10-22 16:11:15 +00:00 |
|
|
Andrej Karpathy
|
48387cd895
|
also bump run1000.sh to new uv sync
|
2025-10-22 16:08:31 +00:00 |
|
|
Andrej
|
2e938530ce
|
delete spurious torch.empty allocation in adamw
fix: remove unnecessary tensor allocation in DistAdamW optimizer
|
2025-10-21 11:35:17 -07:00 |
|
|
Andrej Karpathy
|
a088b7a6ec
|
use enable_gqa of pytorch sdpa, allows us to delete some code, didnt realize it's available
|
2025-10-21 18:07:33 +00:00 |
|
|
Andrej Karpathy
|
94ee507054
|
quick fix base eval due to fewshot requirement
|
2025-10-21 17:56:08 +00:00 |
|
|
Andrej
|
33e8a27f91
|
Merge karpathy/cpu-mps-dev , adding the ability to run on CPU, on MPS, or on CUDA, with autodetect. Gnarly PR, nonzero chance I broke something.
add cpu|mps support
|
2025-10-21 10:26:04 -07:00 |
|
|
Andrej Karpathy
|
50bea28ef9
|
also add readme mention of the cpu mps changes
|
2025-10-21 17:24:48 +00:00 |
|
|
Andrej Karpathy
|
5bdc99abfb
|
merge and resolve conflict
|
2025-10-21 17:19:10 +00:00 |
|
|
Andrej Karpathy
|
dfcb1c16f1
|
Merge branch 'master' into cpu-mps-dev
|
2025-10-21 17:15:53 +00:00 |
|
|
Andrej Karpathy
|
bb71c64579
|
fix silly issue in dataloader, this version is much faster and more portable to mps too
|
2025-10-21 17:12:50 +00:00 |
|
|
karpathy
|
bb786c5560
|
i shouldnt have committed the lock file, i missed that. revert to the flagship build which is linux. sorry to pollute the repo history...
|
2025-10-21 10:07:40 -07:00 |
|
|
Andrej
|
c9ea7a91e2
|
Add customization instructions to README
Added a section on customization for nanochat.
|
2025-10-21 08:57:10 -07:00 |
|
|
Andrej Karpathy
|
03cddd9878
|
actually let's not brick code on git pull. change error to warning
|
2025-10-21 15:13:25 +00:00 |
|
|
Andrej Karpathy
|
fe5aed940b
|
add personality to nanochat. breaks previous code on git pull and requires download of a new file from s3, but there is a helpful error message so hopefully its ok
|
2025-10-21 15:04:58 +00:00 |
|
|
karpathy
|
2e9669e03a
|
upgrading all other files to be able to use cpu/mps as well as cuda. various minor other changes ,e.g. changing max_iterations to num_iterations in sft script for consistency in naming
|
2025-10-20 10:15:17 -07:00 |
|
|
Andrej
|
a09ac812ed
|
toml changes for cpu only install
|
2025-10-20 07:53:15 -07:00 |
|
|
Andrej
|
0f007889dd
|
Add MIT License as a file to the project
|
2025-10-19 17:22:19 -07:00 |
|
|
Andrej
|
5a879f4947
|
export NANOCHAT_BASE_DIR so child processes get it too
|
2025-10-19 17:07:56 -07:00 |
|
|
Andrej Karpathy
|
c1d2ed1c13
|
use orig_model in sampling, silly of me to miss this
|
2025-10-20 00:05:09 +00:00 |
|
|
Andrej Karpathy
|
2bc521a6de
|
use orig_model in sampling, silly of me to miss this
|
2025-10-20 00:04:15 +00:00 |
|
|
Andrej Karpathy
|
9467d83cf2
|
fix memory leak bug in rust tokenizer ty @mitsuhiko
|
2025-10-19 23:54:31 +00:00 |
|
|
AndrejandTancrède Lepoint
|
cf2baf9933
|
fix typo
Co-authored-by: Tancrède Lepoint <tlepoint@users.noreply.github.com>
|
2025-10-17 08:35:41 -07:00 |
|
|
karpathy
|
e4f9b9c64d
|
revert to previous pyproject.toml
|
2025-10-17 08:08:16 -07:00 |
|
|
Andrej
|
e883b1d597
|
Merge pull request #99 from burtenshaw/cpu-mps-dev-ben
Add mps and cpu dependency management
|
2025-10-17 07:24:38 -07:00 |
|
|
karpathy
|
ae02650afe
|
update the midtraining script too
|
2025-10-16 16:33:17 -07:00 |
|
|
karpathy
|
df600b6ed5
|
many small tweaks. base, eval, core work now i think
|
2025-10-16 15:46:18 -07:00 |
|
|
Andrej Karpathy
|
d6d86cbf4c
|
update readme with a link to the CPU|MPS branch
|
2025-10-16 22:03:39 +00:00 |
|
|
Andrej Karpathy
|
ccfe7915ac
|
mention the current d32 chat hosted on nanochat.karpathy.ai, as an example endpoint of the repo
|
2025-10-16 19:32:44 +00:00 |
|
|
karpathy
|
786119d593
|
add autodetect of device and related stuff. getting weird warnings/errors still, so wip
|
2025-10-16 10:26:19 -07:00 |
|
|
karpathy
|
279b74312c
|
adjust comment/guidance on device type
|
2025-10-16 10:06:39 -07:00 |
|
|
karpathy
|
306bc380ab
|
add support for CPU and for MPS. I had to change a few cosmetic things. I also discovered I think a bit of a bug, where I was casting wte to bfloat16 in the wrong place (the model init) instead of in init_weights
|
2025-10-16 10:04:43 -07:00 |
|
|
Andrej Karpathy
|
722da4f543
|
trying to add basic cpu support, will try mps too
|
2025-10-16 16:14:38 +00:00 |
|
|
Andrej Karpathy
|
4346536ab2
|
also allow regenerating assistant message by clicking it, and make sure to feed good seed to generate
|
2025-10-16 01:28:37 +00:00 |
|
|
Andrej Karpathy
|
2846999b8f
|
allow user to click on their message to edit them. conversation after that point is wiped
|
2025-10-16 01:16:22 +00:00 |
|
|
Andrej Karpathy
|
92d52ecc92
|
add slash commands to webui
|
2025-10-16 01:09:53 +00:00 |
|
|
Andrej Karpathy
|
fae3aca951
|
add script to train a 000 version of nanochat. currently it's a bit more like 00 and this would run in probably around 33 hours instead of the budget of 41 hours, so we might tune it later. i think it's ok for now
|
2025-10-15 20:32:22 +00:00 |
|
|
Andrej Karpathy
|
4c3590c499
|
fix subtle issue in token decoding in cases where multiple utf8 bytes need to be emitted into a single codepoint. exampels are emoji or foreign languages. basically we have to accumulate token sequences/text and only emit when we get full codepoints
|
2025-10-15 20:29:54 +00:00 |
|
|
Andrej Karpathy
|
03fa673b7d
|
add basic logging to chat_web, which i think might be fun
|
2025-10-15 19:51:06 +00:00 |
|
|
Andrej Karpathy
|
52bfeea8bd
|
add very basic abuse prevention limits to chat_web so it's ok to host endpoints
|
2025-10-15 19:42:54 +00:00 |
|
|
Andrej Karpathy
|
01fb290f53
|
allow multiple GPUs to do inference in a data parallel way
|
2025-10-15 19:12:19 +00:00 |
|
|
Andrej Karpathy
|
190d9515d0
|
dont evaluate the sampling evals during SFT they are too slow. keep the multiple choice evals. delete unused imports
|
2025-10-15 16:42:23 +00:00 |
|
|
Andrej Karpathy
|
b8076dd367
|
fix bug in learning rate multiplier, it was ramping up instead of ramping down. see more in Issue #68. also add --dry_run option useful for experimentation
|
2025-10-15 16:35:04 +00:00 |
|
|
Andrej
|
67aaca98f5
|
export NANOCHAT_BASE_DIR so child processes get it too
Export the cache directory so that users can use their own cache location
|
2025-10-14 16:01:28 -07:00 |
|
|
Andrej
|
dd6ff9a1cc
|
fix bug in fallback case of find_largest_model
Fix: Handle missing d<number> model tags in find_largest_model
ty
|
2025-10-13 14:38:34 -07:00 |
|
|
Andrej
|
5fd0b13886
|
Merge pull request #2 from epoyraz/patch-1
Update README.md
|
2025-10-13 10:10:15 -07:00 |
|
|
Andrej
|
626bd3e260
|
Add image of the WebUI to readme
|
2025-10-13 08:03:00 -07:00 |
|
|
karpathy
|
da96b46565
|
update link to the new discussion
|
2025-10-13 07:42:09 -07:00 |
|
|
karpathy
|
a53833d04f
|
add nanochat logo png
|
2025-10-13 06:59:59 -07:00 |
|
|
karpathy
|
3a5e0bc50b
|
initial commit
|
2025-10-13 06:49:24 -07:00 |
|