Go to AWS > Route 53 > Hosted zones > sinfrontera.net:
Type A:
Name: api.webapp.sinfrontera.net
Value: your IP
Type CNAME (optional):
Name: www.api.webapp.sinfrontera.net
Value: api.webapp.sinfrontera.net
Cloning the project from GitHub
sudochown-Rubuntu:ubuntu/var/www/sinfronteras
chmod755/var/www/sinfronteras
cd/var/www/sinfronteras/webapp-1
gitclonegit@github.com:adeloaleman/fastapi.git
cdfastapi
vi.env# Don't forget to create the env variables (env variables can also be defined in the systemd service file)
Creating the venv and installing libraries
...
Testing on the terminal
fastapirunmain.py--port8000# from the api directory: fastapi/api/
uvicornapi.main:app--reload--host0.0.0.0--port8000# from the project root directory: fastapi/
gunicorn-w4-kuvicorn.workers.UvicornWorkerapi.main:app--bind0.0.0.0:8000# from the project root directory: fastapi/
Using systemd: A Linux service manager that starts, stops, and monitors long-running processes. We use it to keep the FastAPI app running in the background, start it automatically on reboot, and restart it if it crashes.
sudosystemctldaemon-reload# We run this when we modify the service file (fastapi.service)
sudosystemctlstartfastapi
sudosystemctlenablefastapi
sudosystemctlstatusfastapi
sudojournalctl-ufastapi-n50# To checks log in case of any issues
sudosystemctlrestartfastapi# If we make any changes to the app later
Now we'll use Nginx as a reverse proxy: It sits in front of our FastAPI app so it can be accessed on port 80. Nginx handles client connections and forwards requests to our app running on Gunicorn at port 8000.
sudoln-s/etc/nginx/sites-available/api.webapp.sinfrontera.net/etc/nginx/sites-enabled/
sudonginx-t
sudosystemctlreloadnginx
sudosystemctlrestartnginx# Only if reload fails or Nginx behaves oddly
sudonginx-t
sudosystemctlreloadnginx
sudosystemctlrestartnginx# Only if reload fails or Nginx behaves oddly
To remove the deployment process
sudosystemctlstopfastapi
sudosystemctldisablefastapi
sudorm/etc/systemd/system/fastapi.service
sudosystemctldaemon-reload
sudosystemctlreset-failed
sudorm/etc/nginx/sites-available/api.webapp.sinfrontera.net# We remove the configuration related to our app
sudorm/etc/nginx/sites-enabled/api.webapp.sinfrontera.net
sudonginx-t
sudosystemctlreloadnginx
sudosystemctlrestartnginx# Alternative (full restart, slightly more disruptive)
Deployment Nextjs
Note: In my Ubuntu 18 server tuve que instalar Node.js 16 (que es el más reciente compatible con Ubuntu 18) y Next.js 13 ("next": "^13.5.0") (que es el más reciente compatible con Node.js 16)
Cloning the project from GitHub
gitclonegit@github.com:adeloaleman/nextjs.git
cdnextjs
vi.env# Don't forget to create the env variables
Define the Node.js version via nvm
echo"20.20.0">.nvmrc# Only needed if the file hasn't been created yet, or if you want to use a different version
nvmuse
Installing libraries
npminstall
Prepares our app for production: It compiles, bundles, and optimizes everything so it's ready to deploy.
npmrunbuild
Testing on the terminal
npmstart# Port 3000 by defaultPORT=3001npmstart# If we want to specify the port
PM2 process manager
npminstall-gpm2
# Start Next.js with PM2:
pm2startnpm--name"nextjs"--start# Port 3000 by defaultPORT=3001pm2startnpm--name"nextjs"--start# If we want to specify the port
pm2save# Save the process
pm2startup# Then we have to run the output returned by "pm2 startup". This is to be ran only once. So the first time we deploy an app with PM2. Not for every app.
pm2save# Save again# To verify it worked:
pm2status
pm2list
systemctlstatuspm2-youruser
pm2restartnextjs# If we make any changes to the app later
pm2restartnextjs--update-env# If we make any changes to the app later that included the .env file
Now we'll use Nginx as a reverse proxy: It sits in front of our Next.js app so it can be accessed on port 80. Nginx handles client connections and forwards requests to our app running on PM2 at port 3000.
sudoln-s/etc/nginx/sites-available/webapp.sinfrontera.net/etc/nginx/sites-enabled/
sudonginx-t
sudosystemctlreloadnginx
sudosystemctlrestartnginx# Only if reload fails or Nginx behaves oddly
sudonginx-t
sudosystemctlreloadnginx
sudosystemctlrestartnginx# Only if reload fails or Nginx behaves oddly
To remove the deployment process
pm2stopnextjs# Stop the process
pm2deletenextjs# Remove it from PM2
pm2save# Update saved process list
pm2unstartup# [optional, if we ran pm2 startup]. Remove PM2 from startup. This affects the entire PM2 setup, not just nextjs, so we should NOT run it if we have other apps managed by PM2.
sudovi/etc/nginx/sites-available/default# We remove the configuration related to our app
sudonginx-t
sudosystemctlreloadnginx
sudosystemctlrestartnginx# Alternative (full restart, slightly more disruptive)
End-to-end web application management. Managed service (PaaS) by AWS.
Allows us to automatically create the EC2s, ELB, S3 and Health Monitoring for our application.
We will be utilizing EBN to create the environment we need to deploy our FastAPI application.
EC2 (Elastic Compute Cloud)
Virtual Machines (EC2)
Auto Scaling Groups (ASG)
ELB (Elastic Load Balancing)
Automatically distribute incoming application traf c to your deployed application.
Allows for scalability and traf c distribution.
Health Monitoring:
Make sure our application is healthy and deployed.
Reports when something is wrong, and when you need to take action.
RDS (Relational Database Storage)
Allows you to create databases in the cloud and managed by AWS. Since it is managed by AWS, they will handle provisioning, OS Updates, backups, replications, scalability and more!
If you don't want AWS to handle your database infrastructure, you could create an EC2
Deploy your database on the EC2
Handle all upgrades yourself
Handle all security yourself
Handle all backups yourself
Code Pipeline:
CI/CD for our backend application (FastAPI).
Will connect to our GitHub account to find new pushes to the head.
Automatic deployments to our Elastic Beanstalk.
Amplify
Deploy our SPA applications efficiently (Works with React).
Allow us to implement CI/CD for our front end application.
Store front end environmental variables.
Route53:
Route 53 is AWS scalable Domain Name System (DNS) web service.
ACM (AWS Certificate Manager)
Manage and deploy SSL/TLS certificates. This makes your app go from HTTP -> HTTPS. ACM removes the process of uploading & renewing certificates.
FastAPI deployment
FastAPI Infrastructure: Ésta NO es la mejor representación del la arquitectura final
Create the FastAPI Infrastructure using Elastic Beanstalk EBN
To deploy our FastAPI application from scratch we could create and configure EC2 VMs and an ELB. However, the EBN service (PaaS) allows us to automatically create the environment needed to deploy our FastAPI application. It abstracts the creation of EC2s, ELB, S3, and health monitoring components.
Elastic Beanstalk:
Createte application:
Name: Webapp-fastapi-1
Create environment:
Configure environment:
Environment tier: Web Server environment
Name: Webapp-fastapi-1-env
We can enter a Domain name: webapp-fastapi-1
Platform: Python
Application code: We're gonna leave it as "Sample application" since we're going to implement CI/CD using Code Pipeline
Presets: Single instance (free tier eligible)
Configure service access:
Service role > Create role: This creates an IAM service role so we are giving Elastic Beanstalk permission to create the resources it needs (EC2 instance, S3 bucket, etc)
Trusted entity type: AWS service
Service or use case: Elastic Beanstalk
Use case: Elastic Beanstalk - Environment
Permissions policies: These should be the ones automatically proposed by the menu: AWSElasticBeanstalkEnhancedHealth / AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy
Role name: webapp-fastapi-1-aws-elasticbeanstalk-service-role
EC2 instance profile > Create role: This creates an IAM instance profile with managed policies that allow your EC2 instances to perform required operations.
Trusted entity type: AWS service
Service or use case: Elastic Beanstalk
Use case: Elastic Beanstalk - Compute
Permissions policies: These should be the ones automatically proposed by the menu: AWSElasticBeanstalkMulticontainerDocker / AWSElasticBeanstalkWebTier / AWSElasticBeanstalkWorkerTier
Role name: webapp-fastapi-1-aws-elasticbeanstalk-ec2-role
We'll use AWS CodePipeline to deploy our FastAPI app and implement CI/CD. This will allow us to connect to our GitHub account and automatically deploy code updates to EBN. Every time we update the GitHub repository, the changes will be reflected on EBN.
CodePipeline:
Create Pipeline:
Creation option:
Build custom pipeline
Pipeline settings:
Name: Webapp-fastapi-1-pipeline
Execution mode: Queued
Service role - New service role:
Role name: webapp-fastapi-1-AWSCodePipelineServiceRole-us-west-1
Then we need to go to IAM > Roles > webapp-fastapi-1-AWSCodePipelineServiceRole-us-west-1 > Permissions > Add permission > Attach policies and add AdministratorAccess-AWSElasticBeanstalk
Add source stage:
Source provider: GitHub (via Github App)
Connection > Connect to GitHub: Here we need to make sure we click on "Install a new GitHub app". If we don't do this, the pipeline code won't be automatically updated when we push changes to our GitHub repository. I think is good practice to use "adeloaleman@gmail.com" as connection name
Repository name: adeloaleman/fastapi
Default branch: main
Output artifact format: CodePipeline default
Webhook events: We don't need to add any filter
Add build stage - optional: Skip build stage
Add test stage - optional: Skip test stage
Add deploy stage:
Deploy provider: AWS Elastic Beanstalk
Input artifacts: We can leave as default (SourceArtifact)
Application name: Webapp-fastapi-1
Environment name: Webapp-fastapi-1-env
To avoid error during the deployment, such as "The provided role does not have the elasticbeanstalk:CreateApplicationVersion permission", we need to make sure to attach the AdministratorAccess-AWSElasticBeanstalk managed policy to the webapp-fastapi-1-AWSCodePipelineServiceRole-us-west-1 role. However, it seems that the best and secure way of resolving those errors is by creating a custom inline policy and attaching it to the webapp-fastapi-1-AWSCodePipelineServiceRole-us-west-1 role. For example:
However, the ElasticBeanstalkCreateAppVersion custom inline policy resolved some, but not all, of the errors. Additional permissions appear to be missing.
We also need to remember that the FastAPI project requires a Procfile so that EBN knows how to run the server using Uvicorn. Its main purpose is to declare the command that should be used to start the web application.
Manage and deploy SSL/TLS certificates: Makes SSL/TLS encryption possible; which is a protocol for encrypting internet traffic and verifying server identify. This makes your app go from HTTP -> HTTPS
There are multiple ways of creating SSL/TLS certificates. However, since we have a domain in Route 53, AWS makes this very easy for us.
Go to AWS Certificate Manager > Request a certificate > Request a public certificate:
Domain names:
sinfrontera.net (Fully qualified domain name)
*.webapp.sinfrontera.net
*.sinfrontera.net
Validation method: DNS validation - recommended
Then, go to AWS Certificate Manager > Certificates:
Click on the Certificate ID we just created, which should appear with a "Pending validation" status
Under "Domains" click on "Create records in Route 53"
Select all the domain/subdomains and click "Create records"
After a while, all of our domain/subdomain should appear with a "Success" status
After we have created cerificates for our domain/subdomains, our elastic beanstalk will still only work with HTTP, not HTTPS. This is because:
Our EC2 rules does not yet allow for HTTPS, only HTTP. So, in our EC2 Security Groups we need to created a HTTPS inbound rule.
EC2 intances don't allow HTTPS. So, we need to configure a load balancer
Before configuring the Load balance we'll see "Auto scalling group: Single instance". The Single linstance environement doesn't allow HTTPS. So we need to configure a Load balancer:
Instances: Min: 1 / Max: 1
Apply the changes and wait until the Elastic Beanstalk environment has been updated successfully
We can now go to EC2 > Load Balancers:
You'll see that our Load balancer has been created using 3 AZs. We can modify this to only use 2 so it'll be cheaper. Select the Load Balancer and go to Network mapping > Edit subnets: Here we can just uncheck one of the AZs
Then click on Listener and rules:
We need to add a new listener for the HTTPS protocol (port 443)
Target group: select your target type instance
Default SSL/TLS server certificate: From ACM and we need to select the certificate we've created for our domain
Scroll down to the end and click Add
After this, we'll still see that HTTPS is not reachable. This is becuase we're missing the inbound rule for our EC2
So we go to Security. Click the Security group ID and add an inbound rule for HTTPS
Hosting > Custom domains > Add domain: If our domain is in Route 53, it will be available to be added easily. Otherwise, we'll need to complete some configuration on our domain provider.
Then we just need to configure the subdomain where we want our app to be.
Amplify - Custom domains
Configuring the API_URL environment variable in our backend (fastapi) app
200: OK : Standard Response for a Successful Request. Commonly used for successful Get requests when data is being returned.
201 : The request has been successful, creating a new resource. Used when a POST creates an entity.
204: No : The request has been successful, did not create an entity nor return anything. Commonly used with PUT requests.
3xx : Redirection: Further action must be complete
4xx : Client Errors: An error was caused by the request from the client
400: Bad : Cannot process request due to client error. Commonly used for invalid request methods.
401: Unauthorized : Client does not have valid authentication for target resource
404: Not : The clients requested resource can not be found
422: UnprocessableEntity : Semantic Errors in Client Request
5xx : Server Errors: An error has occurred on the server
500: Internal Server Error : Generic Error Message, when an unexpected issue on the server happened.
Project 1 - A first very basic example
Creating a FastAPI application:
project_1/books.py
fromfastapiimportFastAPI,Bodyapp=FastAPI()BOOKS=[{'title':'Title One','author':'Author One','category':'science'},{'title':'Title Two','author':'Author Two','category':'science'},{'title':'Title Three','author':'Author Three','category':'history'},{'title':'Title Four','author':'Author Four','category':'math'},{'title':'Title Five','author':'Author Five','category':'math'},{'title':'Title Six','author':'Author Two','category':'math'}]@app.get("/books")asyncdefread_all_books():returnBOOKS@app.get("/books/{book_title}")# path parameterasyncdefread_book(book_title:str):forbookinBOOKS:ifbook.get('title').casefold()==book_title.casefold():returnbook@app.get("/books/")# query parameter: http://127.0.0.1:8000/books/?category=scienceasyncdefread_category_by_query(category:str):books_to_return=[]forbookinBOOKS:ifbook.get('category').casefold()==category.casefold():books_to_return.append(book)returnbooks_to_return@app.get("/books/byauthor/")# Get all books from a specific author using path or query parameters asyncdefread_books_by_author_path(author:str):books_to_return=[]forbookinBOOKS:ifbook.get('author').casefold()==author.casefold():books_to_return.append(book)returnbooks_to_return@app.get("/books/{book_author}/")# Using path parameters AND query parametersasyncdefread_author_category_by_query(book_author:str,category:str):books_to_return=[]forbookinBOOKS:ifbook.get('author').casefold()==book_author.casefold()and \
book.get('category').casefold()==category.casefold():books_to_return.append(book)returnbooks_to_return@app.post("/books/create_book")# This end-point doesn't have any parameter but it has a required «body» where we can pass a new book entry, such as {"title": "Title Seven", "author": "Author One", "category": "history"}asyncdefcreate_book(new_book=Body()):BOOKS.append(new_book)@app.put("/books/update_book")# As in the case of the POST method, PUT requires a «body» where we can also pass a book. In this case, if we pass a new book whose title matches some of the existing books, it will be updated. For example: {"title": "Title Six", "author": "Author One", "category": "history"}asyncdefupdate_book(updated_book=Body()):foriinrange(len(BOOKS)):ifBOOKS[i].get('title').casefold()==updated_book.get('title').casefold():BOOKS[i]=updated_book@app.delete("/books/delete_book/{book_title}")asyncdefdelete_book(book_title:str):foriinrange(len(BOOKS)):ifBOOKS[i].get('title').casefold()==book_title.casefold():BOOKS.pop(i)break
Pydantics is used for data modeling, data parsing and has efficient error handling. Pydantics is commonly used as a resource for data validation and how to handle data comming to our FastAPI application.
With Pydantics:
We're gonna create a different request model for data validation.
We're gonna add Pydantics Field data validation on each variable/element of the request body.
HTTP exception is something that we have to raise within our method, which will cancel the functionality of our method and return a status code and a message back to the user so the user is able to know what happened with the request.
We're gonna handle HTTP Exceptions with the HTTPException class from from fastapi
Explicit status code responses:https://www.udemy.com/course/fastapi-the-complete-course/learn/lecture/36994220#overview
We can also add status code responses for a successful API endpoint request. So, so far it will be returned a 200 if the request is a success. Well, a 200 does mean success, but we can go a little bit more detail than just a normal 200 and dictate exactly what status response is returned after each successful.
To do so we're gonna be using from starlette import status. fastAPI is built using Starlet, so Starlet will be installed automatically when you install fastAPI.
project_2/books2.py
fromtypingimportOptionalfromfastapiimportFastAPI,Path,Query,HTTPExceptionfrompydanticimportBaseModel,Fieldfromstarletteimportstatusapp=FastAPI()classBook:id:inttitle:strauthor:strdescription:strrating:intpublished_date:intdef__init__(self,id,title,author,description,rating,published_date):self.id=idself.title=titleself.author=authorself.description=descriptionself.rating=ratingself.published_date=published_dateclassBookRequest(BaseModel):# In project 1 we used Body (from fastapi import Body) as the type of the object that passed in the body of the request. However, using «Body()» we are not able tu add data validations, which can be done with BookRequest(BaseModel)id:Optional[int]=Field(title='id is not needed')title:str=Field(min_length=3)author:str=Field(min_length=1)description:str=Field(min_length=1,max_length=100)rating:int=Field(gt=0,lt=6)published_date:int=Field(gt=1999,lt=2031)classConfig:schema_extra={# This adds an example value that will be displayed in our Swagger. Note that there is no id cause we want the id to be autogenerated'example':{'title':'A new book','author':'codingwithroby','description':'A new description of a book','rating':5,'published_date':2029}}BOOKS=[Book(1,'Computer Science Pro','codingwithroby','A very nice book!',5,2030),Book(2,'Be Fast with FastAPI','codingwithroby','A great book!',5,2030),Book(3,'Master Endpoints','codingwithroby','A awesome book!',5,2029),Book(4,'HP1','Author 1','Book Description',2,2028),Book(5,'HP2','Author 2','Book Description',3,2027),Book(6,'HP3','Author 3','Book Description',1,2026)]BOOKS@app.get("/books",status_code=status.HTTP_200_OK)# By using status.HTTP_200_OK, we dictate exactly what status response is returned after each successful requestasyncdefread_all_books():returnBOOKS@app.get("/books/{book_id}",status_code=status.HTTP_200_OK)asyncdefread_book(book_id:int=Path(gt=0)):# With Pydantic we've added data validation for the post or put request body; but with haven't added any validation for our path parametersforbookinBOOKS:# Now, to add validations for path parameters, we can use the Path() classifbook.id==book_id:returnbookraiseHTTPException(status_code=404,detail='Item not found')# By adding HTTP Exceptions a status code and message is sent back to the user so the user is able to know what happened with the request@app.get("/books/",status_code=status.HTTP_200_OK)asyncdefread_book_by_rating(book_rating:int=Query(gt=0,lt=6)):# Query parameters validation using Query()books_to_return=[]forbookinBOOKS:ifbook.rating==book_rating:books_to_return.append(book)returnbooks_to_return@app.get("/books/publish/",status_code=status.HTTP_200_OK)asyncdefread_books_by_publish_date(published_date:int=Query(gt=1999,lt=2031)):books_to_return=[]forbookinBOOKS:ifbook.published_date==published_date:books_to_return.append(book)returnbooks_to_return@app.post("/create-book",status_code=status.HTTP_201_CREATED)asyncdefcreate_book(book_request:BookRequest):new_book=Book(**book_request.dict())# Here we are converting the book_request object, which is type BookRequest to a type BookBOOKS.append(find_book_id(new_book))deffind_book_id(book:Book):book.id=1iflen(BOOKS)==0elseBOOKS[-1].id+1returnbook@app.put("/books/update_book",status_code=status.HTTP_204_NO_CONTENT)# This is the most common status code for a PUT request. It means, the request was successful but no content is returned to the client.asyncdefupdate_book(book:BookRequest):book_changed=Falseforiinrange(len(BOOKS)):ifBOOKS[i].id==book.id:BOOKS[i]=bookbook_changed=Trueifnotbook_changed:raiseHTTPException(status_code=404,detail='Item not found')@app.delete("/books/{book_id}",status_code=status.HTTP_204_NO_CONTENT)asyncdefdelete_book(book_id:int=Path(gt=0)):book_changed=Falseforiinrange(len(BOOKS)):ifBOOKS[i].id==book_id:BOOKS.pop(i)book_changed=Truebreakifnotbook_changed:raiseHTTPException(status_code=404,detail='Item not found')
We will learn some basic docker commands that will help us in improving our workflow.
Section 4: MongoDB Basics
We will learn very basic MongoDB commands and we'll execute them inside the docker container and in Pycharm Professional.
Section 5: Web API Project Structure
In this section we'll learn how to structure the project and we will write some basic endpoints with FastAPI just to make you more familiar with writing endpoints.
Section 6: Storage Layer
We talk about CRUD and we'll apply the repository pattern to develop and In-Memory repository and a MongoDB repository in order to use them within our application. We will also test the implementations.
Section 7: Movie Tracker API
We will write the actual API for tracking movies using the previous developed components. We'll implement the application's settings module and we'll add pagination to some of the routes. In the end we will write unit tests.
Section 8: Middleware
We'll talk about Fast API middleware and how to write your own custom middleware.
Section 9: Authentication
We'll talk about implementing Basic Authentication and validating JWT tokens.
Section 10: Deployment
We'll containerize the application and we will deploy it on a local microk8s kubernetes cluster. In the end we'll visualise some metrics with Grafana. Having metrics is a good way to observe your applications performance and behaviour and troubleshoot it.
For testing the API we are going to use Insomnia, which is a powerful REST client that allows developers to interact with and test RESTful APIs. It provides a user-friendly interface for making HTTP requests, inspecting responses, and debugging API interactions. The tool is often used for tasks such as sending requests, setting headers, managing authentication, and viewing API documentation.
To install it on Ubuntu:
sudosnapinstallinsomnia
Creating our Mongo Docker comtainer using docker-compose
Or we can use our favory DB IDE to manage our MongoDB. I'm using MongoDB for VS Code extension.
Let's see some MongoDB basics:
playground-1.mongodb.js
show('databases')// show databases can be used from mongoshuse('movies')// use movies can be used from mongosh. If the movies collection doesn't exits it will create itdb.movies.insertOne({'title':'My Movie','year':2022,watched:false})db.movies.insertMany([{'title':'The Shawshank Redemption','year':1994,'watched':false},{'title':'The Dark Knkght','year':2008,'watched':false},{'title':'Puld Fiction','year':1994,'watched':false},{'title':'Fight Club','year':1999,'watched':false},{'title':'The Lord of the Rings: The Two Towers','year':2002,'watched':false},])db.movies.findOne()// Find all the moviesdb.movies.find()// Filtering by titledb.movies.find({'title':'Fight Club'})// Find a movie that has been produced before 2000db.movies.find({'year':{'$lt':2000}})// Filter by iddb.movies.find({'_id':ObjectId('647b582d74a86f2cb913d881')})// Find all movies but skip the first one and limit the result to only 2 moviesdb.movies.find().skip(1).limit(2)// Sortingdb.movies.find().sort({'year':-1})// 1: ascending; -1: descending// Select only some attributesdb.movies.find({},{'title':1,'year':1})db.movies.find({},{'title':0})// Delete a documentdb.movies.deleteOne({'_id':ObjectId('647b55cdc98722c8abe8ee94')})db.movies.find()// Updatingdb.movies.updateOne({'_id':ObjectId('647b582d74a86f2cb913d87e')},{$set:{'watched':true}})db.movies.updateOne({'_id':ObjectId('647b582d74a86f2cb913d87e')},{$inc:{'year':-3}})db.movies.find()
