与「EyeCandy」相关的搜索结果

That deep V is doing the absolute most 😍🔥 Gray top barely holding on while those curves steal the whole scene… cleavage this perfect should be illegal. #CleavageGoals# #DeepV# #CurvesForDays# #SexySelfie# #HotGirlSummer# #Thicc# #BustyBeauty# #EyeCandy#

今天看到的最漂亮的网站： 把 k8s 集群做成了机架的样式，而且高度可操作，真的是 eye candy，还分了 front back 和 side 三个面，每个面讲的是同一套集群的不同切面，是一件精致的艺术品。 注意上面的各种按钮和场景都是可交互的，用来学习概念也很棒，我已经把玩许久了，谁做的这么棒啊。

好漂亮的 tui， 连终端看盘都出来了，这是我看到的做的比较复杂的 tui 了吧，用了一下，纯纯 eye candy，不过比较多功能都是要收费的，适合终端硬核玩家。桌面端的设计也很漂亮，不拘一格。

又做了一个只为自己爽的 eye candy，做了一个库用来渲染一颗复古 disco 球，多种风格和参数，最近好像 disco 风开始火起来，突然来了灵感。 AI 彻底解放了我有设计想法但是不会实现的问题。最近做的几个库都是这样的，虽然最终效果没有那么完美，但是让我看的爽就可以了。

In 2019, OpenAI announced GPT-2 with this post: Today (~5 years later) you can train your own for ~$672, running on one 8XH100 GPU node for 24 hours. Our latest llm.c post gives the walkthrough in some detail: Incredibly, the costs have come down dramatically over the last 5 years due to improvements in compute hardware (H100 GPUs), software (CUDA, cuBLAS, cuDNN, FlashAttention) and data quality (e.g. the FineWeb-Edu dataset). For this exercise, the algorithm was kept fixed and follows the GPT-2/3 papers. Because llm.c is a direct implementation of GPT training in C/CUDA, the requirements are minimal - there is no need for conda environments, Python interpreters, pip installs, etc. You spin up a cloud GPU node (e.g. on Lambda), optionally install NVIDIA cuDNN, NCCL/MPI, download the .bin data shards, compile and run, and you're stepping in minutes. You then wait 24 hours and enjoy samples about English-speaking Unicorns in the Andes. For me, this is a very nice checkpoint to get to because the entire llm.c project started with me thinking about reproducing GPT-2 for an educational video, getting stuck with some PyTorch things, then rage quitting to just write the whole thing from scratch in C/CUDA. That set me on a longer journey than I anticipated, but it was quite fun, I learned more CUDA, I made friends along the way, and llm.c is really nice now. It's ~5,000 lines of code, it compiles and steps very fast so there is very little waiting around, it has constant memory footprint, it trains in mixed precision, distributed across multi-node with NNCL, it is bitwise deterministic, and hovers around ~50% MFU. So it's quite cute. llm.c couldn't have gotten here without a great group of devs who assembled from the internet, and helped get things to this point, especially ademeure, ngc92, @gordic_aleksa, and rosslwheeler. And thank you to @LambdaAPI for the GPU cycles support. There's still a lot of work left to do. I'm still not 100% happy with the current runs - the evals should be better, the training should be more stable especially at larger model sizes for longer runs. There's a lot of interesting new directions too: fp8 (imminent!), inference, finetuning, multimodal (VQVAE etc.), more modern architectures (Llama/Gemma). The goal of llm.c remains to have a simple, minimal, clean training stack for a full-featured LLM agent, in direct C/CUDA, and companion educational materials to bring many people up to speed in this awesome field. Eye candy: my much longer 400B token GPT-2 run (up from 33B tokens), which went great until 330B (reaching 61% HellaSwag, way above GPT-2 and GPT-3 of this size) and then exploded shortly after this plot, which I am looking into now :)

Anyone else find themselves estimating the "GPT grade" of things you hear/read? When something is poorly written or generic, it's "GPT-2 grade" content. When something is lit, you can complement it as being "GPT-7 grade" etc. This reminds me of a fun side project I had saved for myself but will realistically never get around to, maybe someone can take a shot. Simply - train a classifier that predicts GPT-grade of any text. The training data would be samples from models of increasing strength. It might be that GPT models are too coarse and that too much changed between each one. Ideally you'd want a nice miniseries where everything is held constant except the model size, e.g. Llama 3 series, esp when they also release the smaller (and bigger!) models. Sample from the models over many prompts (or use base models?), classify the model size, then point it at various text on the internet, e.g. study the divergence between the comments section of WSJ and VC thought leadership :p. To be clear I have no idea if this would work, e.g. the classifier might very well latch on to the style a lot more than the content. Or it might measure not exactly an "intelligence" of text, but more just a "generic-ness", a proxy for frequency or so. It might also be an interesting way to study what is learned as you increase model size. But that's why it's an interesting project - it feels like it might kind of work, but it's not obvious and a number of details are tbd. Eye candy: ChatGPT attempts to visualize the above

🔥llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention) On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32, with no flash attention yet, and slightly stale PyTorch (2.1.0). - It is a direct implementation of the training loop and backpropagation in C/CUDA. - It compiles and runs instantly. No more "hit run then wait for tens of seconds for unknown reasons", for mountains of inscrutable abstractions to build a Universe. - It deletes the need for the Python interpreter and a deep learning library. - It allocates all the memory a single time at the start. - It's pretty cool. How: Getting this to work required us to write a lot of custom CUDA kernels, and doing this manually (instead of using Tensor ops of aten/PyTorch and torch.compile etc.) is a bit like programming in assembly. And you spend quality time looking at more assembly (CUDA PTX/SASS). But this also means we get to hyperoptimize the code and possibly explore optimizations that torch.compile might find difficult to, which is awesome. Examples of optimizations that went in over the last few days: - we're being clever with our memory consumption in the backward pass, only using a few buffers we need to propagate the gradients, saving memory capacity. - one fused classifier kernel does the last layer forward pass, the loss, and kicks off the backward pass. - many improvements to all the kernels involved, including e.g. gains from carefully constraining execution within the autoregressive mask in attention - cuBLAS(Lt) calls for all heavy lifting matmuls, and fused bias accumulation Big credits to two CUDA experts who appeared from somewhere on the internet to help this open source project, ngc92 and ademeure. We're hanging out of Github and Discords of CUDAMODE and my NN Zero to Hero. Next steps: - more optimizing of our (fp32) kernels, and especially switch to flash attention. - mixed precision training (fp16 to start). - multi-gpu training (DDP to start). - data & evals to set up a proper GPT-2 training runs - 🚀 repro GPT-2 (1.6B) training run. - more modern architectures etc. (Llama 3?) - writing, videos, exercises on building all of this from scratch. Figure 1: eye candy: timing profile of the kernels (one layer). NVIDIA cutlass kernels with solid compute throughput taking up a lot of the running time => nice.