forked from karpathy/nanoGPT
-
Notifications
You must be signed in to change notification settings - Fork 17
Issues: ReaLLMASIC/nanoGPT
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Experiments: Rotary Position Embedding Swapping/Finetuning for HW Effiiciency
#228
opened Aug 14, 2024 by
gkielian
4 tasks
Quest: Add Benchmarks and Pretokenized JSON for training
epic
Epic level contrib
#222
opened Aug 9, 2024 by
gkielian
Exploration Automation: Obtain Dataset, Select Tokenization, and Cache results
#219
opened Aug 8, 2024 by
gkielian
Sample.py: fix inference for p50k_base and cl100k_base
bug
Something isn't working
#213
opened Jul 31, 2024 by
gkielian
Quest: Add a Rough draft for Mixture of Experts
enhancement
New feature or request
epic
Epic level contrib
good first issue
Good for newcomers
help wanted
Extra attention is needed
#140
opened Apr 9, 2024 by
gkielian
Quest: Create a training efficient version of windowed attention for experimenting with long contexts
enhancement
New feature or request
epic
Epic level contrib
good first issue
Good for newcomers
#139
opened Apr 9, 2024 by
gkielian
Quest: Curriculum and Preprocessing for Translation Targets
enhancement
New feature or request
epic
Epic level contrib
good first issue
Good for newcomers
help wanted
Extra attention is needed
Quest: Have run experiments print a number of combinations and optional printing of all options to file
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
#126
opened Mar 17, 2024 by
gkielian
Quest: Add Quantization Aware Training and Inference
enhancement
New feature or request
epic
Epic level contrib
help wanted
Extra attention is needed
Quest: Explore training stability of network with bf16 and fp16
enhancement
New feature or request
good first issue
Good for newcomers
#118
opened Mar 9, 2024 by
gkielian
Quest: Add Quad2 and Relu2 Softmax Variations
enhancement
New feature or request
good first issue
Good for newcomers
#104
opened Feb 29, 2024 by
gkielian
Add Learned Channel Embeddings
enhancement
New feature or request
#103
opened Feb 28, 2024 by
gkielian
Add minbpe support for tokenization
enhancement
New feature or request
#102
opened Feb 28, 2024 by
gkielian
prepare.py script: add progress bar for tokenizers
enhancement
New feature or request
good first issue
Good for newcomers
#96
opened Feb 26, 2024 by
gkielian
Previous Next
ProTip!
Find all open issues with in progress development work with linked:pr.