Tech With Team, Ollama

Question 1

today I’m going to show you how to access

Answer 1

Today, I’m going to show you how to access some of the best models that exist completely for free and locally on your own computer [2].

Question 2

we’re going to be doing that in just

Answer 2

We’re going to be doing that in just a few very simple lines of code using Hugging Face and Langchain [2].

Question 3

I’m going to show you how to use

Answer 3

I’m going to show you how to use various Hugging Face models for free inside of a Python application, and then we’ll connect them to Langchain so we can build something even more interesting [2, 3].

Question 4

with that said let’s get into the

Answer 4

With that said, let’s get into the video, but first, some important information [2].

Question 5

understand we’re going to be using something called

Answer 5

Understand, we’re going to be using something called Transformers [4].

Question 6

now this is a free open source package

Answer 6

Now, this is a free, open-source package that you can install with Python using pip, if you’re familiar with that [4].

Question 7

it allows you to utilize tons of different

Answer 7

It allows you to utilize tons of different models that come from Hugging Face [4].

Question 8

hugging face is a platform you can see

Answer 8

Hugging Face is a platform that has all kinds of open-source models available [4].

Question 9

some of them you do need to pay

Answer 9

Some of them you do need to pay for, or you need some specific access, but most of them are completely free, and you can just download them and use them on your own computer [4].

Question 10

what the Transformers package does is just make

Answer 10

What the Transformers package does is just make it extremely easy to access these free Hugging Face models and to download them and use them on your computer [5].

Question 11

you’ll see here that in just a

Answer 11

You’ll see here that in just a few lines of code, you can run pretty much any of these models, assuming you have the correct hardware [5].

Question 12

if you wanted to go further you could

Answer 12

If you wanted to go further, you could even fine-tune these models and add a bunch of advanced configurations [5].

Question 13

we’re going to be using Transformers and we’re

Answer 13

We’re going to be using Transformers, and we’re going to use this in combination with something called Langchain [3, 5].

Question 14

now Lang chain is another python package that

Answer 14

Now, Langchain is another Python package that just makes it a lot easier to work with LLMs [3].

Question 15

so if you wanted to use one of

Answer 15

So, if you wanted to use one of these Transformer models, you can connect this with Langchain and then add things like memory, connect multiple models together, and I’ll show you the basics of that in this video [3].

Question 16

now we’re going to go ahead and start

Answer 16

Now, we’re going to go ahead and start getting all of this set up [3].

Question 17

first things first we’re just going to

Answer 17

First things first, we’re just going to make an account on Hugging Face [3, 6].

Question 18

this is because a lot of these models

Answer 18

This is because a lot of these models require that you accept some terms, or their license agreement, before you’re able to actually download them and use them [6].

Question 19

so go to hugging face I’ll leave this

Answer 19

So, go to Hugging Face; I’ll leave this link in the description [6].

Question 20

you should be able to go up here

Answer 20

You should be able to go up here somewhere in the top right-hand corner and just make a new account [6].

Question 21

once you make a new account we’re going

Answer 21

Once you make a new account, we’re going to go to access token; we’re just going to leave this page open, and we’ll come back to it in one second [6].

Question 22

to get started here we need to

Answer 22

To get started here, we need to set up our environment to be able to execute these different Python packages [6, 7].

Question 23

now this does take a second so just

Answer 23

Now, this does take a second, so just bear with me [7].

Question 24

once the environment is configured then it’s

Answer 24

Once the environment is configured, then it’s very easy to write the code [7].

Question 25

now you'll notice that what I'm using

Answer 25

Now, you'll notice that what I'm using here is **PyCharm** [7].

Question 26

anyways you can use whatever IDE that you

Answer 26

Anyways, you can use whatever IDE that you want, but if you do want to check this out again, I have those links below [8].

Question 27

so first things first we want to

Answer 27

So, first things first, we want to create that virtual environment [9].

Question 28

so what I'm going to do is open

Answer 28

So, what I'm going to do is open up my terminal here, and from my terminal, I'm going to execute a command that will create this environment for me [9].

Question 29

now you do need python installed in order

Answer 29

Now, you do need Python installed in order for this to work [9].

Question 30

what the virtual environment will do is

Answer 30

What the virtual environment will do is just create an isolated area where we can have the dependencies specifically for this project [9, 10].

Question 31

so in order to do that we're going

Answer 31

So, in order to do that, we're going to type `python -DM venv`, and then `venv` [10].

Question 32

what this does is use VV to create

Answer 32

What this does is use VV to create a new virtual environment called venv or VV, and then once this step is finished, we're going to activate that virtual environment [10].

Question 33

the next step is to activate the virtual

Answer 33

The next step is to activate the virtual environment so that we're using it when we install various dependencies [10, 11].

Question 34

next step we're going to make a new

Answer 34

Next step, we're going to make a new file inside of our directory called `requirements.txt` [12].

Question 35

now this is where we're going to

Answer 35

Now, this is where we're going to put the various requirements that we need in order to work with the **Transformers** and **Langchain** Library [12].

Question 36

now that you have the requirements.txt file

Answer 36

Now that you have the `requirements.txt` file created from your virtual environment, you're going to type the command `pip install dashr` and then `requirements.txt` [13].

Question 37

okay so all of that's been installed

Answer 37

Okay, so all of that's been installed, and the next step here is to simply get our **Hugging Face** token [13, 14].

Question 38

now we need the hugging face token because

Answer 38

Now, we need the **Hugging Face** token because a lot of the models just require that you accept some license agreement; they're free to use, but you have to essentially check a box, and that's connected to your **Hugging Face** account [14].

Question 39

so if you try to pull certain models

Answer 39

So, if you try to pull certain models, it will give you an error, and that's because you haven't added the token [14].

Question 40

so what we're going to do is

Answer 40

So, what we're going to do is, from our user access tokens again, you can just go to **Hugging Face**, create a new account, press on your profile, press here on access tokens, and simply create a new one [14, 15].

Question 41

now this command will work as long

Answer 41

Now, this command will work as long as you've installed **Transformers** in your ual environment [16].

Question 42

okay so we're almost done here that

Answer 42

Okay, so we're almost done here; that is pretty much the environment setup [16, 17].

Question 43

okay so first thing we're going to

Answer 43

Okay, so first thing we're going to do here is we're going to say from **Transformers** import and then we're going to import pipeline [17, 18].

Question 44

now pipeline is a simplified way of running

Answer 44

Now, pipeline is a simplified way of running various models [18].

Question 45

what I'm doing is I'm using this thing

Answer 45

What I'm doing is I'm using this thing called pipeline; pipeline just automatically sets everything up for me, and all I have to do is specify the task [19].

Question 46

the first step is it's going to download

Answer 46

The first step is it's going to download this; it's going to take a second to download [20].

Question 47

once the model is downloaded it will

Answer 47

Once the model is downloaded, it will be stored on your machine, and then you can use it very quickly after that [21].

Question 48

now you'll also notice that we're getting

Answer 48

Now, you'll also notice that we're getting some kind of warnings; I'll show you how to mute those in one second, and you'll notice that it says that the device is using CPU [21].

Question 49

now this is the main thing I want

Answer 49

Now, this is the main thing I want to focus on: how do we get this to use your GPU? [21].

Question 50

so first let's look at how we

Answer 50

So, first, let's look at how we can run these models using our GPU [22].

Question 51

now in order to do that you

Answer 51

Now, in order to do that, you are going to need an Nvidia GPU, or at least that's what I'm going to recommend because that's what works the best [22].

Question 52

now in order for your Nvidia GPU

Answer 52

Now, in order for your Nvidia GPU to be used for this, you need to download and install CUDA [22, 23].

Question 53

once Cuda is downloaded what you're going

Answer 53

Once Cuda is downloaded, what you're going to want to do is restart any terminal instance that you have, so close your IDE, reopen it, and you're going to type the following command just to verify that it's installed [23, 24].