ExpressJS Notes
Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications. It is an open source framework developed and maintained by the Node.js foundation.
Below is a simple example of express app :
NOTE : You can use "nodemon" package to avoid running the server again and again to see file changes.
---------------------------------------------------------------------------------------------------------------
Routing in Express
Routing refers to how an server side application responds to a client request to a particular endpoint. This endpoint consists of a URI (a path such as / or /books) and an HTTP method such as GET, POST, PUT, DELETE, etc.
Routes can be either good old web pages or REST API endpoints. In both the cases the general syntax used to create routes in Express is the following :
- The 'app' is an instance of the express module
- METHOD is an HTTP request method (GET, POST, PUT or DELETE)
- PATH is a path of the route on the server.
- HANDLER is the request handler function executed when the route is matched.
Request Handler
A request handler is a function that will be executed every time the server receives a particular request defined by HTTP method. It accepts 2 parameters request and response object. The request handler is a callback function that processes the request and write to the response object.
The incoming request data is acessed through the "Request Object", whereas the outgoing data that is to be returned from the route function is converted into "Response Object" before sending to client.
The Request object (req) represents the HTTP request and has properties for the request query string, parameters, body, HTTP headers, and so on.
NOTE : Find all the parameters & functions that come with request and response objects with official doc - https://expressjs.com/en/4x/api.html#req
The Response object (res) specifies the HTTP response which is sent by an Express app when it gets an HTTP request. It sends response back to the client browser. It facilitates you to put new cookies value and that will write to the client browser.
Request-Response Cycle
The cycle of operations that get executed starting with a request hitting the Express application till a response leaves the application is called the request-response cycle.
The request-response cycle in express is as followed :
- First a user gives a client a URL, the client builds a request for information to be generated by a server.
- When the server receives that request, it uses the information included in the request to build a response that contains the requested information.
- Once built, that response is sent back to the client in the requested format, to be rendered to the user.
---------------------------------------------------------------------------------------------------------------
Express Response Methods
When an express application receives a request from client,it gets passed the request and response object. There are a bunch of methods available to send appropriate response back to the client :
- res.send()
- res.json()
- res.end()
- res.download()
- res.status()
- res.redirect()
1] res.send()
This is a general function which sends the HTTP response. This function accepts a single parameter body that describe the HTTP 'body' which is to be sent in the response.
The signature of this method looks like this: res.send([body]) where the body can be any of the following: Buffer, String, an Object and an Array.
This method performs many useful tasks for simple non-streaming responses. For example, it automatically assigns the Content-Length & Content-Type HTTP response header field (unless previously defined). When the parameter is a String, the method sets the Content-Type to “text/html”. When the parameter is an Array or Object, Express responds with the JSON representation.
2] res.json()
This method sends a response (with the correct content-type) that is the parameter converted to a JSON string using JSON.stringify(). It sends a JSON response. This method is identical to res.send() when an object or array is passed, but it also converts other data types (non-objects) to json format. The res.json will convert non objects (ex. null, undefined etc) as well which are actually not a valid JSON whereas res.send will not convert them.
There is no actual difference between res.send and res.json, both methods are almost identical. res.json calls res.send at the end before using JSON.stringify().
3] res.end()
When you want to respond with some data,you use res.json() or res.send() , but when you want to end the response without any data use res.end() method. We use res.end() if we want to end the response without providing any data. This could be useful for a 404 page.
NOTE : The Header variable content-type is empty and content-length is 0, since we are not responsing any data to the client.
4] res.download()
This function transfers the file at path as an “attachment”. Typically, browsers will prompt the user for download. The method invokes the callback function fn(err) when the transfer is complete or when an error occurs.
If the callback function is specified and an error occurs, the callback function must explicitly handle the response process either by ending the request-response cycle, or by passing control to the next route.
5] res.status()
This function Sets the HTTP status for the response object.
6] res.redirect()
This function redirects to the URL derived from the specified path, with specified status, a positive integer that corresponds to an HTTP status code . If not specified, status defaults to “302 “Found”.
Redirects can be a fully-qualified URL for redirecting to a different site, or can be relative to the root of the host name. For example, if the application is on http://example.com/admin/post/new, we can redirect to the URL http://example.com/admin
---------------------------------------------------------------------------------------------------------------
Dynamic URL/ Route parameters
Rather than setting up static URLs, we can also make a URL dynamic by using route parameters. Route parameters are basically data variables that we pass along with the URL.
NOTE : The captured values are populated in the req.params object, with the name of the route parameter specified in the path as their respective keys.
---------------------------------------------------------------------------------------------------------------
Useful : 1] Click
Express.Router()
When creating route paths we often list of of them in a single file,but if we have a large number of routes defined then it becomes a hassle to manage them. The Express.router() function allows us to encapsulate routes in another file and then include them in our main file,this way we can keep our code modular and easily manage routes.
The express.Router() function is used to create a new router object. This function is used when you want to create a new router object in your program to handle requests.
In below example we write our routes in another file and later add them to a particular given path.
User.js
main.js
NOTE : In the above path we attach the external routes to the root path ("/") , but we can attach it to any new path. For example we we attach it to "/ABC" then all external paths will be attached like this : /ABC/User
---------------------------------------------------------------------------------------------------------------
Sending Files in Response
The res.sendFile() method transfers the file present on the server at the given path. Sets the Content-Type response HTTP header field based on the filename’s extension. You can any type of file like images,audio,html etc. All the files sent will be rendered directly on the browser upon receiving.
NOTE : Unless the root option is set in the options object, path must be an absolute path to the file. When the root option is provided, the path argument is allowed to be a relative path.
The method invokes the callback function fn(err) when the transfer is complete or when an error occurs. If the callback function is specified and an error occurs, the callback function must explicitly handle the response process
NOTE : The res.sendFile() is different than res.render() .The render method works when you have a templating engine. The sendfile method, on the other hand, simply sends a given file to the client, regardless of the type and contents of the file.
---------------------------------------------------------------------------------------------------------------
Useful : 1] Click
Render HTML with Template engine
When you want to send static files which do not change like Images,Videos,HTML files then you can use the res.sendFile() method. But if we want to render different values based on variables passed then we have to use a templating engine,in that case we can use the res.render() function.
Some popular template engines that work with Express are Pug, Mustache, and EJS. The Express application generator uses Jade as its default, but it also supports several others. Avoid using pug or jade since their syntax is a bit different than HTML.
NOTE : Before rendering an HTML template you have to set the path to the "Views" directory which will contain all HTML template files and also set the template engine to be used for rendering.
NOTE : Unlike flask and FastAPI where the file extensions for our HTML templates remain the same,in express when we use different template engines,the file extension may also change. Eg - In EJS engine the templates have to be named by the extension ".ejs" rather than ".html".
In the below example we render the file "index.ejs" which is inside the "./views" folder and also set some data for placeholders.
main.js
.views/index.ejs
---------------------------------------------------------------------------------------------------------------
Sending Cookies in Response
HTTP is a stateless protocol, which means that the server can’t distinguish whether a user is visiting the site for the first time or not. So to solve this problem, sites use cookies.
When a Client visits a particular site for the first time, the client will not have any cookies set by the site. So the server creates a new cookie and sends it to the Client machine itself. So in the next subsequent visits, the client machine will attach the cookie to the request header and send it. The server then retrieves the cookies from the request object and uses that cookie information to identify the user and maintain the session.
1] res.cookie(name,value,options)
This function sets cookie name to value. The value parameter may be a string or object converted to JSON. The options parameter is an object that can be used to configure other cookie related things like expiry date, httpOnly etc.
NOTE : Once you have set the cookie value,that value will be sent with all the subsequent requests from the client. You can acess it again from the request body (req.cookies.key)
2] res.clearCookie(name,options)
This function clears the cookie specified by name. Web browsers and other compliant clients will only clear the cookie if the given options is identical to those given to res.cookie(), excluding expires and maxAge.
---------------------------------------------------------------------------------------------------------------
Express Middlewares
A Middleware is a function that is executed during the request-response cycle and have access to request object (req) and response object (res). Middleware is executed during the window between when a server receives a request and when it sends a response.
Express middleware can be built-in or from a third party. Since Express.js has limited functionality of its own, an Express app is largely comprised of multiple middleware function calls. You could write your own middleware for Express.js, but most developers prefer to use and configure built-in and third-party tools for common tasks.
Middleware Chaining
Middleware can be chained from one to another, Hence creating a chain of functions that are executed in order. These middlewares are chained like a stack,the first added middleware is executed first and so on. The last function sends the response back to the browser.
Middleware functions take 3 following arguments :
req: the request objectres: the response objectnext:nextis a function that passes execution to the next middleware when it is done.
Middlewares are used inside the app.use() or app.METHOD() functions, where METHOD refers to the HTTP verbs GET, POST, PUT, DELETE, and more.
The next() function in express is responsible for calling the next middleware function in the chain (if there is one). Every middleware function takes this next() function as an argument and executes it at the end, except the last middleware function in the chain.
NOTE : Every route we create using app.METHOD() function takes a callback function as argument,this callback function is also a middleware but its the last middleware in the chain,hence there is no use even calling the next() function inside it since there is not middleware after it.
Analogy of middlewares
To understand how middleware works, imagine you own a lemonade stand where customers bring their own lemons and you make the lemonade. You’re responsible for evaluating the lemons’ origin and freshness, discarding any subpar lemons, and, finally, making the lemonade.
To reduce your workload, you hire a worker — we’ll call him Larry — to make sure the lemons were grown organically and without any harmful chemicals. In this analogy, Larry is the middleware that functions between you and your customers’ lemons.Now you’re making a profit, so you hire two other employees, Curly and Moe. Larry checks the lemons’ origin and passes the organically grown lemons to Curly, who discards the rotten lemons and hands the good ones to Moe. Moe verifies their freshness and hands the fresh lemons to you.
Below is an example middleware chain we created. Each middleware is executed in the same order that we added them.
The app.use() function
The app.use() function is used to mount one or more middleware functions at the particular route path. When a request comes for a particular route the middlewares mounted at that particular path will be executed. The app.use() takes 2 arguments,which are the following :
1] Path (Default is root or "/") - The path for which the middleware function is invoked; can be any of :- A string representing a path.
- A path pattern.
- A regular expression pattern to match paths.
- An array of combinations of any of the above.
2] Callback - Callback middleware functions,this can be one of following :
- A middleware function.
- A series of middleware functions (separated by commas).
- An array of middleware functions.
- A combination of all of the above.
If we specify a path in app.use() then the middleware will get executed only when a request is sent to that particular route path.
We can also pass multiple middlewares as a series,array or a combination of both.
We can also insert the middlewares directly inside the app.METHOD(), rather than using app.use() function.
---------------------------------------------------------------------------------------------------------------
Types of Middlewares
An Express application can use the following types of middleware :
- Application-level middleware
- Router-level middleware
- Error-handling middleware (we'll see this later)
- Built-in middleware
- Third-party middleware
Application-level middleware
Application level middleware are bound to an instance of express, using app.use() and app.METHOD(). They are not tied to any particular route and are executed everytime any request comes to the server regardless of the route they come to.
In the below example we did'nt specify any "path" for middleware,hence the middleware gets executed on all the routes,so everytime a request comes to server the middlewares gets executed.
Router-level middleware
---------------------------------------------------------------------------------------------------------------
Built-in middlewares
Express has the following built-in middleware functions :
- express.static - It serves static assets such as HTML files, images, and so on.
- express.json - It parses incoming requests with JSON payloads.
- express.urlencoded - It parses incoming requests with URL-encoded payloads.
1] express.static()
This is a built-in middleware function in Express. It is used to serves static files like Images,videos,HTML etc. You can use the express.static middleware to make it possible to access files from a specific folder via HTTP.
const express = require('express');
const app = express();
// The first parameter to `express.static()` is the directory to serve.
app.use(express.static('./public'));
app.listen(3000);For example if there is a css file inside a folder named public then,with the above script, you can open http://localhost:3000/home.css in your browser and see the CSS file.
NOTE : When a file is not found, instead of sending a 404 response, it instead calls next() to move on to the next middleware, allowing for stacking and fall-backs. It is recommended to add this middleware at the end of your express app.
2] express.json()
This is a built-in middleware function in Express. It parses incoming requests with JSON payloads. After parsing,a new 'body' object containing the parsed data is populated on the request object (i.e. req.body). You can then acess the JSON data by using the keys on body object.
It expects the data to be received as JSON and with content-type as "Json/application".
NOTE : If there was no body to parse, then it may be that the Content-Type was not matched, or an error occurred.
NOTE : You need express.json() and express.urlencoded() for POST and PUT requests only, because in both these requests you are sending data to the server and you are asking the server to accept or store that data (object), which is enclosed in the body (i.e. req.body) of that (POST or PUT) Request.
---------------------------------------------------------------------------------------------------------------
Third-party middlewares
There are number of third-party express middlewares which people have created. We can use them in our express app just like we use other middlewares. Some commonly used express middlewares :
- morgan
- Helmet
- Cors
- Express-rate-limit
- Cookie-parser
- session
1] Morgan Logger
Morgan is an HTTP request level Middleware. It is a great tool that logs the requests along with some other information depending upon its configuration and the preset used. It proves to be very helpful while debugging and also if you want to create Log files.
You can either create your own formats for logging the required information or use the pre-defined formats available. There are 5 predefined formats that you can utilise to quickly obtain the information you require :
- combined - This sets your logs to the Apache standard combined format
- common - Refers to the Apache common format standard
- dev - A log format that is colour-coded (based on request status)
- short - Less than the normal format, with only a few items you'd expect to see in a request logline
- tiny - Even less, simply the reaction time and a few extras
2] Express Rate Limit
The rate-limiting feature makes it possible to secure the Backend API from malicious attacks. It allows us to cap the number of requests that a user can make to our APIs. The express-rate-limit is a third party module to implement rate limiting in our express apps.
In the below example we limit the number of request to 5 within 12 hour window.
The rate-limit object takes the following variables :
windowMsis the window size. In our case, we used a 24 hours window duration in milliseconds.maxis the maximum amount of requests a user can make within a given window duration.messageis the response message that a user gets whenever they have exceeded the limit. May be astring, JSON object, or any other value that Express's response.send method supports.headersindicates whether to add headers to show the total number of requests and the duration to wait before trying to make requests again.
3] Helmet
When creating API's its very important to secure HTTP headers,as they contain information related to your application and anyone can use that to find vulnerabilities in your application.Helmet helps you secure your Express apps by setting various HTTP headers.
Helmet.js is a collection of various Node modules that interface with Express. Each module provides configuration options for securing different HTTP header values. The top-level helmet function is a wrapper around 15 smaller middlewares.
// Includes all 15 middlewares
app.use(helmet());
app.use(helmet.contentSecurityPolicy());
app.use(helmet.crossOriginEmbedderPolicy());
app.use(helmet.crossOriginOpenerPolicy());
app.use(helmet.crossOriginResourcePolicy());
app.use(helmet.dnsPrefetchControl());
app.use(helmet.expectCt());
app.use(helmet.frameguard());
app.use(helmet.hidePoweredBy());
app.use(helmet.hsts());
app.use(helmet.ieNoOpen());
app.use(helmet.noSniff());
app.use(helmet.originAgentCluster());
app.use(helmet.permittedCrossDomainPolicies());
app.use(helmet.referrerPolicy());
app.use(helmet.xssFilter());
NOTE : It is advised to use Helmet middleware in every express app to increase security. You can easily modify or remove any middleware from inside the top-level function.
For some HTTP headers, Helmet.js automatically defaults to the “secure” option. Others, like Content-Security-Policy, require the developer to make an explicit configuration. This is usually because the "best practice" may break functionality or degrade user experience -- so configurations must be tuned accordingly.
4] Express-Session (Useful : 1] Click)
When the client makes a login request to the server, the server will create a session and store it on the server-side. When the server responds to the client, it sends a cookie. This cookie will contain the session’s unique id stored on the server, which will now be stored on the client. This cookie will be sent on every request to the server. We use this session ID and look up the session saved in the database or the session store to maintain a one-to-one match between a session and a cookie. This will make HTTP protocol connections stateful.
The session data is serialized as JSON when stored, you can set data or acess session data using the session object inside the request object (req.session). We store the session id in a cookie, and keep the data server-side. The client will receive the session id in a cookie, and will send it along with every HTTP request. We’ll reference that server-side to associate the session id with the correct session data stored locally.
NOTE : The only required parameter the session object takes is the "secret", which is used to encrypt/decrypt session Id's to prevent session-hijacking attacks.
There are many other optional parameters,some important ones are listed on the official documentation : http://expressjs.com/en/resources/middleware/session.html
---------------------------------------------------------------------------------------------------------------
Create REST API with Express
When you expect a payload in Json format,you have to use the Json parser middleware to convert the json into request body variables.
---------------------------------------------------------------------------------------------------------------
Query Strings & Parameters
In simple terms, a query string is the part of a URL (Uniform Resource Locater) after the question mark (?). It is meant to send small amounts of information to the server via the url. This information is usually used as parameters to query a database, or maybe to filter results. The query parameters are the actual key-value pairs.
While query parameters are typically used in GET requests, it's still possible to see them in POST and DELETE requests, among others. Your query parameters can be retrieved from the query object on the request object sent to your route. It is in the form of an object in which you can directly access the query parameters you care about.
In the example above, we assume the username and password parameters always exist. If any parameter is not given a value then its undefined.
---------------------------------------------------------------------------------------------------------------
Error-Handling Middleware
When writing error handling code inside express apps,we may end up handling individual errors inside each route using try/catch, this would get too redundant quickly and wouldn’t scale well as you add more and more routes. Below is an example.
A much better option would be to leverage Express’s middleware functions here. You could write one or more middleware functions for handling errors in your application that all of your routes could utilize.
The error handling middleware are defined in the same way as other middleware functions, except that error-handling functions MUST have 4 arguments instead of three – (err, req, res, next). Till now we were handling errors in the routes itself. The error handling middleware allows us to separate our error logic and send responses accordingly. The next() method we discussed in middleware takes us to next middleware.
For error handling, we have to pass the error to next() function,like this - next(err). A call to this function skips all middleware and matches us to the next error handler for that route. Below is an modified version of previous example doing the same error handling but now with middlewares.
Instead of defining the handling behavior inside each route, we place all our logic inside the middleware. Then, based on the kind of error, we can modify the error object (or throw a custom error) and accordingly deal with it in the middleware.
NOTE : Always add the error handling middlewares at the end after app.use() and other middlewares.
In the previous section, we worked with just one middleware to handle all our errors. However, in practice, multiple middleware functions are usually employed for different type of error handling to have further abstractions. For example, one middleware for logging errors, another for responding to the client, perhaps another as a fail-safe catch-all handler, etc.
---------------------------------------------------------------------------------------------------------------
Comments
Post a Comment