-
-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' of github.com:gist-rs/book
- Loading branch information
Showing
42 changed files
with
512 additions
and
206 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Hugging Face - Candle | ||
|
|
||
| > [https://github.com/huggingface/candle]() | ||
| A minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use. | ||
|
|
||
| ### Examples | ||
|
|
||
| These online demos run entirely in your browser: | ||
|
|
||
| - [yolo](https://huggingface.co/spaces/lmz/candle-yolo): pose estimation and | ||
| object recognition. | ||
| - [whisper](https://huggingface.co/spaces/lmz/candle-whisper): text to speech. | ||
| - [LLaMA2](https://huggingface.co/spaces/lmz/candle-llama2): text generation. | ||
| - [LLaMA and LLaMA-v2](https://github.com/huggingface/candle/blob/main/candle-examples/examples/llama/): general LLM. | ||
| - [Falcon](https://github.com/huggingface/candle/blob/main/candle-examples/examples/falcon/): general LLM. | ||
| - [StarCoder](https://github.com/huggingface/candle/blob/main/candle-examples/examples/bigcode/): LLM specialized to code | ||
| generation. | ||
| - [Quantized LLaMA](https://github.com/huggingface/candle/blob/main/candle-examples/examples/quantized/): quantized version of | ||
| the LLaMA model using the same quantization techniques as | ||
| [llama.cpp](https://github.com/ggerganov/llama.cpp). | ||
|
|
||
| <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/quantized/assets/aoc.gif" width="600"> | ||
|
|
||
| - [Stable Diffusion](https://github.com/huggingface/candle/blob/main/candle-examples/examples/stable-diffusion/): text to | ||
| image generative model, support for the 1.5, 2.1, and SDXL 1.0 versions. | ||
|
|
||
| <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/stable-diffusion/assets/stable-diffusion-xl.jpg" width="200"> | ||
|
|
||
| - [yolo-v3](https://github.com/huggingface/candle/blob/main/candle-examples/examples/yolo-v3/) and | ||
| [yolo-v8](https://github.com/huggingface/candle/blob/main/candle-examples/examples/yolo-v8/): object detection and pose | ||
| estimation models. | ||
|
|
||
| <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.od.jpg" width="200"><img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.pose.jpg" width="200"> | ||
|
|
||
| - [segment-anything](https://github.com/huggingface/candle/blob/main/candle-examples/examples/segment-anything/): image | ||
| segmentation model with prompt. | ||
|
|
||
| <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/segment-anything/assets/sam_merged.jpg" width="200"> | ||
|
|
||
| - [Whisper](https://github.com/huggingface/candle/blob/main/candle-examples/examples/whisper/): speech recognition model. | ||
| - [Bert](https://github.com/huggingface/candle/blob/main/candle-examples/examples/bert/): useful for sentence embeddings. | ||
| - [DINOv2](https://github.com/huggingface/candle/blob/main/candle-examples/examples/dinov2/): computer vision model trained | ||
| using self-supervision (can be used for imagenet classification, depth | ||
| evaluation, segmentation). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| # Async | ||
|
|
||
| > 🤔 refer to [The State of Async Rust](https://corrode.dev/blog/async/) | ||
| ## The State | ||
|
|
||
| ```mermaid | ||
| graph LR; | ||
| A("Async madness") | ||
| A-.Try to cover Rust standard.->C("async-std<br/>abandoned 😱") | ||
| A-.Try to cover any runtime.->B("Tokio<br/>Multi threaded by default = complex 🫠") | ||
| ``` | ||
|
|
||
| ## Async vs Threads | ||
|
|
||
| ### Threads | ||
|
|
||
| ```rust,editable | ||
| use std::error::Error; | ||
| use std::fs::File; | ||
| use std::io::Read; | ||
| use std::path::Path; | ||
| use std::{thread, time}; | ||
| // This fn could be used by both async and sync callers, | ||
| // eliminating the need for an asynchronous runtime. | ||
| fn read_contents<T: AsRef<Path>>(file: T) -> Result<String, Box<dyn Error>> { | ||
| let mut file = File::open(file)?; | ||
| let mut contents = String::new(); | ||
| file.read_to_string(&mut contents)?; | ||
| return Ok(contents); | ||
| } | ||
| fn main() { | ||
| thread::scope(|scope| { | ||
| // worker thread 1 | ||
| scope.spawn(|| { | ||
| let contents = read_contents("foo.txt"); | ||
| // do something with contents | ||
| }); | ||
| // worker thread 2 | ||
| scope.spawn(|| { | ||
| let contents = read_contents("bar.txt"); | ||
| // ... | ||
| }); | ||
| // worker thread 3 | ||
| scope.spawn(|| { | ||
| let contents = read_contents("baz.txt"); | ||
| // ... | ||
| }); | ||
| }); | ||
| // Threads get joined automatically | ||
| } | ||
| ``` | ||
|
|
||
| ### Async | ||
|
|
||
| ```rust | ||
| #[tokio::main] | ||
| async fn main() { | ||
| // This will print a warning, but compile and do nothing at runtime | ||
| read_contents("foo.txt"); | ||
| // futures do nothing unless you `.await` or poll them | ||
| } | ||
| ``` | ||
|
|
||
| > 🤔 refer to [Asynchronous Programming in Rust](https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html#async-vs-threads-in-rust) | ||
|  <span class="speech-bubble">If you don't need async for performance reasons, threads can often be the simpler alternative.</span> | ||
|
|
||
| ## Best practices | ||
|
|
||
| 1. `synchronous` by default. | ||
| 1. opt into `async` as needed. | ||
| 1. less main with `#[tokio::main]`. | ||
|
|
||
| ```rust,no_run | ||
| #[get("/")] | ||
| fn index() -> impl Response { | ||
| let users = db::get_users(); | ||
| Response::ok().body(render_template("index.html", users)) | ||
| } | ||
| // A route, which profits from concurrent IO | ||
| // It sends multiple requests to an external API and aggregates the results | ||
| // Note how the function itself is does not need to be async | ||
| #[get("/users")] | ||
| fn users(count: usize) -> impl Response { | ||
| // Start a local, single-threaded runtime with smol's async-executor | ||
| let rt = smol::LocalExecutor::new(); | ||
| // Run the async code on the runtime | ||
| let results = rt.run(async { | ||
| let mut results = Vec::new(); | ||
| for id in 0..count { | ||
| let result = reqwest::get(format!("https://api.example.com/users/{}", id)).await?; | ||
| results.push(result); | ||
| } | ||
| Ok(results) | ||
| }); | ||
| Response::ok().body(render_template("users.html", results)) | ||
| } | ||
| // This does not need to be async either! | ||
| // In the background, it might use a thread pool to handle multiple requests | ||
| fn main() -> Result<()> { | ||
| let app = App::new() | ||
| .mount("/", index) | ||
| .mount("/users", users) | ||
| .run(); | ||
| } | ||
| ``` | ||
|
|
||
|  <span class="speech-bubble">Read more 👉 [Asynchronous Programming in Rust](https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html)</span> |
Oops, something went wrong.