Jae-Kyung Cho Being unique is better than being perfect

MLOps study - Raviraja Week 5: Docker

24 Oct 2022

Week 5 is about Docker.




Fast api

ref: Asynchronous programming using Async The 5 methods of REST API

FastAPI lets you run a model via HTTP communication. It’s a library used not only for running ML models but also for implementing various APIs. If you have some understanding of web development and HTTP communication (REST API), it’ll probably be a bit easier to grasp. It’s a very convenient way to turn a model into a web service, so studying just this a little more isn’t a bad idea!!

# Week5.py file
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
import uvicorn
from NLPmodel import ColaPredictor

app = FastAPI(title="MLOps Basic App")
predictor = ColaPredictor("models/model.onnx")

@app.get("/", response_class=HTMLResponse)
async def home():
    html = """
    <html>
        <h2>This is a sample NLP Project</h2>
    </html>
    """
    return HTMLResponse(content=html, status_code=200)

@app.get("/predict")
async def get_prediction(text: str):
    result = predictor.predict(text)
    return result

if __name__=='__main__':
    uvicorn.run("Week5:app", host="0.0.0.0", port=8000)

FastAPI provides a swagger document page. (Instead of using the API via URL, you can use a GUI like a supervise page!) It’s perfect for checking whether things work.

localhost:8000/docs

In the code above, to actually use the API directly, you’d write it like this. (%20 means a space.)

localhost:8000/predict?text=Hi%20My%20name%20is%20Jack

So now, if we just make a web page, it means we can connect the URL and use it!!




Docker

Docker is the most famous container management tool. It has the advantage that the whale is incredibly cute, but it also seems to be very widely used because it’s offered for free. Docker can be seen as broadly composed of three parts (Docker file, Docker image, Docker container).

A Docker file stores various information such as dependencies, some code, how to run it, and so on. A Docker image refers to a lightweight, standalone, executable software package, storing the entire code, runtime, libraries, etc. A Docker container is an instance running a Docker image; you can think of a program as running inside it. If we express this as a diagram, it looks like this. Screenshot 2022-11-16 12:37:07 AM



Docker file

The kinds of content that can be written inside a Docker file are as follows.

  • Which OS was used
  • Which dependencies were installed
  • How to compile the code Besides these, various other pieces of information can be included.

One of the best advantages of a Docker file is that once this information is built it gets cached, so if a change occurs during a rebuild, it simply rebuilds only the changed part. That saves time accordingly.

First, create a file named Dockerfile.

touch Dockerfile

Let’s put the following content inside.

FROM huggingface/transformers-pytorch-cpu:latest
COPY ./ /app
WORKDIR /app
RUN pip install -r requirements_inference.txt
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
EXPOSE 8000
CMD ["uvicorn", "Week5:app", "--host", "0.0.0.0", "--port", "8000"]

(In case a UTF-8 related error pops up, refer to here.)

Let’s look at each a bit more.

Command Description  
From base image, fetched from Docker hub. It’s common to fetch the ubuntu os with something like ubuntu:latest.
COPY Among the local files, the files that will also be added to the container and used. Above, it adds /app and the entire current subfolder to the container.  
WORKDIR working directory  
RUN Specifies which command to run in the container. It’s common to install dependencies.  
EXPOSE Specifies which port to leave open.  
CMD Specifies which commands will be included when running the container for the very first time.  

So in the case of the Dockerfile written above, it’s based on the docker image provided by huggingface, installs the dependency libraries via requirements.txt, and runs the FastAPI-related process via CMD.

Now, if you build the Docker file, a Docker image is created.

docker build -t <Docker file REPOSITORY>:<Docker file TAG> .

Since the Repository and Tag are what get uploaded to docker hub, you can make them custom or use existing ones.



Docker image

Let’s check the built Docker image.

docker images

Screenshot 2022-11-15 11:41:19 PM

(To delete an image? docker rmi [ID])



Docker container

Now let’s use a Container to create an instance of the docker image we made.

docker run -it -p 8000:8000 --name <container name> <docker image repository>:<docker image tag>

Then the FastAPI thing we used above runs exactly as it did!!!!!X100

Let’s briefly look at the most commonly used options when using docker run, as below. (source)

Option Description
-it Option for terminal input/output
-p Option for port forwarding (connects the host’s and container’s ports)
–name Set the container name
-e Set environment variables inside the container
-d Background mode
-v Mount a host directory into the container
-rm Automatically delete the container when the process ends
–link Connect containers

I’ll also record several other commands worth knowing. Someday there’ll be a day I look at these again…

Command Description
docker rm Delete the container with that name
docker ps Check the currently running containers
docker ps -a Check the remaining containers (containers that ended with an error remain here)
docker rmi $(docker images -f “dangling=true” -q) Delete the containers that ended with an error
docker create Create a container with that name (does not start the process)
docker start Start the container with that name (usable when created with create)
docker stop Stop a running container
docker detach/attach Run the container in the background / go back inside the container
ctrl+p, ctrl+q docker detach shortcut
docker exec Run inside the container



Compose file

Installing Docker compose 
sudo apt-get install docker-compose-plugin

Using a Docker compose file, you can easily run multiple dockers. When I properly get into MLOps later, it seems I’ll also have to study communication between containers…!! Let’s create a docker-compose.yaml file.

# docker-compose.yaml
version: '3'
services:
  prediction_api:
    build: .
    container_name: 'inference_container'
    ports:
      - '8000:8000'

And

docker compose up

you can run it by executing this command. If you put multiple docker definitions in Services, they all run at once!! (Here there’s only one, though~)




Download py file Download NLP model file Download requirements.txt

references:

Comments

Related posts