![PyTorch](/img/default-banner.jpg)
- Видео 361
- Просмотров 3 034 822
PyTorch
Добавлен 13 авг 2019
Welcome to the official PyTorch RUclips Channel. Learn about the latest PyTorch tutorials, new, and more.
PyTorch is an open source machine learning framework that is used by both researchers and developers to build, train, and deploy ML systems that solve many different complex challenges. PyTorch is an open source project at the Linux Foundation.
PyTorch is an open source machine learning framework that is used by both researchers and developers to build, train, and deploy ML systems that solve many different complex challenges. PyTorch is an open source project at the Linux Foundation.
Accelerating LLM family of models on Arm Neoverse based Graviton AWS processors with KleidiAI
In this webinar we will introduce changes that were made to PyTorch to improve performance of LLaMA family of models on AArch64. To achieve performance improvements we have introduced two new ATen operations torch.ops.aten._kai_weights_pack_int4() and torch.ops.aten._kai_input_quant_mm_int4() that are using highly optimised packing and GEMM kernels that are available in KleidiAI library. These two new PyTorch operators are leveraged by gpt-fast to firstly, quantize weights to INT4 by using symmetric per-channel quantization and add additional array containing quantization scales and secondly, dynamically quantize activation matrix and execute INT8 matrix multiplication of activation matri...
Просмотров: 631
Видео
Official PyTorch Documentary: Powering the AI Revolution
Просмотров 174 тыс.2 месяца назад
This film unveils the authentic narrative of PyTorch’s inception, attributing its existence to a dedicated group of unsung heroes driving technological innovation. The documentary shares the strength of the PyTorch community, resonating with our communities across the globe. We hope this story of PyTorch inspires greater contributions, attracts more contributors to the project, and fosters wide...
PyTorch Docathon Kickoff H1 2024
Просмотров 8333 месяца назад
Welcome to the kickoff for the PyTorch Docathon, June 2024! This event is dedicated to enhancing the quality of the PyTorch documentation with the invaluable assistance of our community. Our hope with this Docathon is to simplify the process for new users to get started with PyTorch, guide them in effectively utilizing its features, and ultimately expedite the transition from research to produc...
PyTorch Documentary Trailer
Просмотров 11 тыс.9 месяцев назад
Here's a sneak peek of the PyTorch Documentary in collaboration with Speakeasy Productions. Here from key founders and innovators of the PyTorch project from companies like AMD, AWS, Meta and Microsoft. Full documentary to be released in early 2024.
PyTorch Docathon H2 2023
Просмотров 1,8 тыс.10 месяцев назад
Welcome to the kickoff for our second PyTorch Docathon! This event will be happening virtually November 1-November 15, 2023. The Docathon is a hackathon-style event focused on improving the documentation by enlisting the help of the community. For more details, check out our blog: pytorch.org/blog/announcing-docathon-h2-2023/
Introducing ExecuTorch from PyTorch Edge: On-Device AI... - Mergen Nachin & Orion Reblitz-Richardson
Просмотров 2,7 тыс.10 месяцев назад
Introducing ExecuTorch from PyTorch Edge: On-Device AI Stack and Ecosystem, and Our Unique Differentiators - Mergen Nachin & Orion Reblitz-Richardson, Meta Speakers: Orion Reblitz-Richardson, Mergen Nachin This high-level presentation focuses on the technological advancements in PyTorch Edge, our on-device AI stack. We will provide an overview of the current market landscape and delve into PyTo...
Lightning Talk: Enhancements Made to MPS Backend in PyTorch for Applications Running... - Kulin Seth
Просмотров 1,3 тыс.10 месяцев назад
Lightning Talk: Enhancements Made to MPS Backend in PyTorch for Applications Running on Mac Platforms - Kulin Seth, Apple Since PyTorch 2.0, MPS backend has qualified for “beta” stage which provides wider operator support (300 ) and network coverage. We will provide details about new features introduced in MPS backend such as how to add custom operations to your network and profiling applicatio...
Lightning Talk: Harnessing NVIDIA Tensor Cores: An Exploration of CUTLASS & OpenAI..- Matthew Nicely
Просмотров 2,7 тыс.10 месяцев назад
Lightning Talk: Harnessing NVIDIA Tensor Cores: An Exploration of CUTLASS & OpenAI..- Matthew Nicely
Lightning Talk: TorchFix - a Linter for PyTorch-Using Code with Autofix Support - Sergii Dymchenko
Просмотров 1,1 тыс.10 месяцев назад
Lightning Talk: TorchFix - a Linter for PyTorch-Using Code with Autofix Support - Sergii Dymchenko
Lightning Talk: PyTorch 2.0 on the ROCm Platform - Douglas Lehr, AMD
Просмотров 4,8 тыс.10 месяцев назад
Lightning Talk: PyTorch 2.0 on the ROCm Platform - Douglas Lehr, AMD
Lightning Talk: State of PyTorch - Alban Desmaison, Meta - Speakers: Alban Desmaison
Просмотров 3,4 тыс.10 месяцев назад
Lightning Talk: State of PyTorch - Alban Desmaison, Meta - Speakers: Alban Desmaison
What's New for PyTorch Developer Infrastructure - Eli Uriegas & Omkar Salpekar
Просмотров 1,3 тыс.10 месяцев назад
What's New for PyTorch Developer Infrastructure - Eli Uriegas & Omkar Salpekar
Lightning Talk: Triton Compiler - Thomas Raoux, OpenAI
Просмотров 7 тыс.10 месяцев назад
Lightning Talk: Triton Compiler - Thomas Raoux, OpenAI
PyTorch Korea User Group: The Beginning, Present, and Future - Junghwan Park
Просмотров 53010 месяцев назад
PyTorch Korea User Group: The Beginning, Present, and Future - Junghwan Park
Lightning Talk: AOTInductor: Ahead-of-Time Compilation for PT2 Exported Models - Bin Bao, Meta
Просмотров 1,1 тыс.10 месяцев назад
Lightning Talk: AOTInductor: Ahead-of-Time Compilation for PT2 Exported Models - Bin Bao, Meta
Keynote: How PyTorch Became the Foundation of the AI Revolution - Joe Spisak, Product Director, Meta
Просмотров 55210 месяцев назад
Keynote: How PyTorch Became the Foundation of the AI Revolution - Joe Spisak, Product Director, Meta
Lightning Talk: Exploring PiPPY, Tensor Parallel and Torchserve for Large... - Hamid Shojanazeri
Просмотров 54810 месяцев назад
Lightning Talk: Exploring PiPPY, Tensor Parallel and Torchserve for Large... - Hamid Shojanazeri
Lightning Talk: The Fastest Path to Production: PyTorch Inference in Python - Mark Saroufim, Meta
Просмотров 1,6 тыс.10 месяцев назад
Lightning Talk: The Fastest Path to Production: PyTorch Inference in Python - Mark Saroufim, Meta
Lightning Talk: PT2 Export - A Sound Full Graph Capture Mechanism for PyTorch - Avik Chaudhuri, Meta
Просмотров 66510 месяцев назад
Lightning Talk: PT2 Export - A Sound Full Graph Capture Mechanism for PyTorch - Avik Chaudhuri, Meta
Lightning Talk: Large-Scale Distributed Training with Dynamo and... - Yeounoh Chung & Jiewen Tan
Просмотров 76110 месяцев назад
Lightning Talk: Large-Scale Distributed Training with Dynamo and... - Yeounoh Chung & Jiewen Tan
Lightning Talk: Profiling and Memory Debugging Tools for Distributed ML Workloads on GPUs- Aaron Shi
Просмотров 1,5 тыс.10 месяцев назад
Lightning Talk: Profiling and Memory Debugging Tools for Distributed ML Workloads on GPUs- Aaron Shi
Keynote: Welcome & Opening Remarks - Ibrahim Haddad, Executive Director, PyTorch Foundation
Просмотров 19310 месяцев назад
Keynote: Welcome & Opening Remarks - Ibrahim Haddad, Executive Director, PyTorch Foundation
Lightning Talk: Accelerating LLM Training on Cerebras Wafer-Scale... - Mark; Natalia; Behzad & Emad
Просмотров 66710 месяцев назад
Lightning Talk: Accelerating LLM Training on Cerebras Wafer-Scale... - Mark; Natalia; Behzad & Emad
Keynote: AMD & PyTorch: A Powerful Combination for Generative AI - Negin Oliver
Просмотров 41110 месяцев назад
Keynote: AMD & PyTorch: A Powerful Combination for Generative AI - Negin Oliver
Lightning Talk: Accelerating PyTorch Performance with OpenVINO - Yamini, Devang & Mustafa
Просмотров 61510 месяцев назад
Lightning Talk: Accelerating PyTorch Performance with OpenVINO - Yamini, Devang & Mustafa
Lightning Talk: Lessons from Using Pytorch 2.0 Compile in IBM's Watsonx.AI Inference - Antoni Martin
Просмотров 21610 месяцев назад
Lightning Talk: Lessons from Using Pytorch 2.0 Compile in IBM's Watsonx.AI Inference - Antoni Martin
Lightning Talk: Accelerating Inference on CPU with Torch.Compile - Jiong Gong, Intel
Просмотров 1,2 тыс.10 месяцев назад
Lightning Talk: Accelerating Inference on CPU with Torch.Compile - Jiong Gong, Intel
Lightning Talk: Accelerated Inference in PyTorch 2.X with Torch...- George Stefanakis & Dheeraj Peri
Просмотров 1,8 тыс.10 месяцев назад
Lightning Talk: Accelerated Inference in PyTorch 2.X with Torch...- George Stefanakis & Dheeraj Peri
Lightning Talk: Building Intermediate Logging for PyTorch - Kunal Bhalla, Meta
Просмотров 41410 месяцев назад
Lightning Talk: Building Intermediate Logging for PyTorch - Kunal Bhalla, Meta
Lightning Talk: Streamlining Model Export with the New ONNX Exporter - Maanav Dalal & Aaron Bockover
Просмотров 89710 месяцев назад
Lightning Talk: Streamlining Model Export with the New ONNX Exporter - Maanav Dalal & Aaron Bockover
this guy's credentials makes one feel so inadequate
Thank you for creating this content!
this documentary is clear evidence of how communistic use of means of production to share innovations work best than capitalistic competition trying to destroy the adversary in the market
Technology goes too fast for me to keep up. I know some Python and now I just learn PyTorch exist and it’s the next BIG thing. I have the impression that only the first people who created it and use it may have some future from it.
GAN does not stand for generative autoencoders but generative adversery network
Polska gurom
Omg, he speak accurately and so fast , he's smart
Thank you PyTorch. Greetings from Popayan, Colombia.
Great explanation
very smart man
❤❤❤❤❤🎉🎉🎉
Thank you Pytorch. Greetings from Popayan, Colombia.
brings back memories from Facebook AI days
now shill anything other than nvidia
Good news: Digitalism is killing capitalism. A novel perspective, first in the world! Where is capitalism going? Digitalism vs. Capitalism: The New Ecumenical World Order: The Dimensions of State in Digitalism by Veysel Batmaz is available for sale on Internet.
is there a sample colab?
I didn't get much speedup using this.
Please don’t bake the captions into the video. If I cannot understand them, please put the transcription into the toggled closed captions. It’s distracting when not necessary like when someone is actually speaking a different language than the rest of the video (English in this case).
skill issue
PyTorch is easily one of the best frameworks I've ever used for any purpose. From the bottom of my heart and all the other little people that you've empowered to do great things: Thank you. ❤
Great visualization!🥰
Good stuff, It would be awesome to upload the video in full hd!
I always love Edward Yang stuff. Keep up the good work boys 💪
One of the most impactful things ive watched on RUclips
Microsoft and Nvidia are at the forefront of the AI revolution. SmythOS can help you tap into this power to drive innovation in your business. #AI #SmythOS
Thank you. Good guide
*Summary* * *(**00:00:12**)* *Torch Compile Missing Manual:* This document is a comprehensive guide to troubleshooting and optimizing PyTorch model compilation using `torch.compile`. It's meant to help you resolve issues and get the best performance from compiled models. * *(**00:01:14**)* *Structured for Troubleshooting:* The manual is organized around common problems (e.g., compiler crashes, slow compile time, poor performance, high memory usage). Each section dives into potential causes and diagnostic steps. * *(**00:02:18**)* *Key Debugging Strategies:* * *(**00:02:28**)* *Torch Trace:* This tool provides a comprehensive overview of the PyTorch 2 compilation process. It's highly recommended for identifying bottlenecks and issues. Share torch traces in bug reports for easier debugging. * *(**00:04:56**)* *Ablations (Isolating Components):* Use the `torch.compile` API to disable specific parts of the compilation pipeline (e.g., inductor, aot_autograd) to pinpoint the source of problems. * *(**00:07:36**)* *Focus on Performance:* A dedicated section helps you analyze and improve performance. Start with the "Start Here" section to check for common performance pitfalls. * *(**00:10:19**)* *Tools and Techniques:* * *(**00:10:23**)* *Profiling (PyTorch Profiler and Chrome Trace):* Profile your compiled model to understand where time is spent. Look for opportunities to reduce kernel launch overhead and improve kernel execution time. * *(**00:12:23**)* *Inspecting Generated Code (Teal Parse):* Examine the generated code (Trident kernels) to identify inefficiencies or potential problems with the compilation process. * *(**00:14:21**)* *Case Studies:* Learn from real-world examples of performance debugging and optimization. * *(**00:14:53**)* *Collaboration and Feedback:* Share new performance challenges or missing information with the PyTorch team to improve the manual and `torch.compile` itself. *In essence, this manual aims to empower PyTorch users to effectively troubleshoot and optimize the performance of their compiled models through various diagnostic strategies, tools, and collaborative feedback.* I used Google Gemini 1.5 Pro exp 0801 to summarize the transcript. Cost (if I didn't use the free tier): $0.1490 Time: 91.64 seconds I added a 61 second delay to prevent a rate limit of the free tier. Input tokens: 39863 Output tokens: 901
Love this Documentary! Thank you a lot!
hi! could you please upload an example for multi-CPU only (without GPU training)?
Thanks! could you please give a tutorial for using only multiple CPUs?
Thank you Intel and Pytorch. Greetings from Popayan, Colombia.
One of the best explanations I have been able to find. Thank you for your time and effort!
Thank you so much for this incredible documentary on PyTorch! It's amazing to see how PyTorch has become a cornerstone in the AI and machine learning community. The depth of insight into its development and impact is truly inspiring. PyTorch has revolutionized the way researchers and developers approach deep learning. Its dynamic computational graph allows for more flexibility and ease in building and modifying complex models, which is a significant advancement over static graph frameworks. This has made it easier for many in the field to experiment, iterate, and advance their research. The documentary highlights PyTorch’s role in powering cutting-edge research and applications, from natural language processing to computer vision. It’s impressive to see how it has facilitated breakthroughs in various domains, enabling innovations that were previously out of reach. I also appreciate the focus on the community aspect of PyTorch. The collaborative nature of the open-source project has brought together a diverse group of contributors, driving continuous improvement and support for the framework. Thank you to everyone involved in creating this documentary and for all the hard work that has gone into developing PyTorch. Your contributions are making a significant impact on the future of AI and machine learning! Thanks again for sharing this valuable resource! ^^
Artificial intelligence is built in Python.
██████╗ ██╗ ██╗████████╗██╗ ██╗ ██████╗ ███╗ ██╗ ██╔══██╗╚██╗ ██╔╝╚══██╔══╝██║ ██║██╔═══██╗████╗ ██║ ██████╔╝ ╚████╔╝ ██║ ███████║██║ ██║██╔██╗ ██║ ██╔═══╝ ╚██╔╝ ██║ ██╔══██║██║ ██║██║╚██╗██║ ██║ ██║ ██║ ██║ ██║╚██████╔╝██║ ╚████║ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝
Meta IDE
Boring
I'm student of AI in med and I love MONAI. A convenient tool for the end2end pipeline
Very bad idea using dark theme for the code, hard to see. Bad resolution also
Thank you for not being afraid to let us go behind the scenes with your development! At JetSoftPro, a software development service, we admire your focus on your audience and understanding of the market you're working for
God I love this documentary !
jokingly,.... fun people hey! ... should these guys run the world? After watching this people ... I am even stronger about technology is everyone's business. Loudly, comes to me Community Consultation and CONSENT.
I am unable to install "datautils" and getting the following error. do you know python-3 compatible "datautils"??? Collecting datautils Using cached datautils-1.0.3.tar.gz (25 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [7 lines of output] Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 34, in <module> File "/tmp/pip-install-z0y75pgf/datautils_626ab6395b7241ef98af8cc89e8623e8/setup.py", line 366 print 'adding', sub_package ^^^^^^^^^^^^^^^^^^^^^^^^^^^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)? [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.
We actually have a documentary on a Python Library and more than 100k people watching it in just a month of release. This is amazing guys.
what kind of IDE is this at 21:57 ? using an unregistered version at AMD 😮
Where can i find quality code examples?
I think the entire point is missed. The acceleration that LLMs offer is the ability to access and distill information at an unprecedented rate
@11:35 LeNet model for the 32X32 one channel images. import torch import torch.nn as nn import torch.nn.functional as F class LeNet(nn.Module): def __init__(self): super(LeNet, self).__init__() self.conv1 = nn.Conv2d(1, 6,3) self.conv2 = nn.Conv2d(6, 16, 3) self.fc1 = nn.Linear(16*6*6, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84,10) self.relu = F.relu def forward(self , x): x = self.conv1(x) #input shape 32X32 -> ouput shape 30X30 x = self.relu(x) x = F.max_pool2d(x ,2) #input shape 30X30 -> ouput shape 15X15 x = self.conv2(x) #input shape 15X15 -> ouput shape 13X13 x = self.relu(x) x = F.max_pool2d(x ,2) #input shape 13X13 -> ouput shape 6X6 x = torch.flatten(x) x = self.fc1(x) x = self.relu(x) x = self.fc2(x) x = self.relu(x) x = self.fc3(x) return x net = LeNet() input_image = torch.rand(1, 1, 32, 32) output = net(input_image) output
@8:00 Set requires_grad = True example: x = torch.rand(1, 10, requires_grad = True)
@7:14 r = (torch.rand(2,2) - 0.5) * 2 # normalize values to -1. to 1. r = torch.rand(2,2) - 0.5 * 2 # generates random values in range of [-1., 0.] @8:00 Set requires_grad = True example: x = torch.rand(1, 10, requires_grad = True)
Pytorch truly brought modularity in AI and built a reproducible ecosystem.