Building a Simple Web Server with ExpressJS

This lecture will be a bit different this week and instead of covering a specific chapter topic, we will walk through building a simple ExpressJS web server for you to expand upon later in this course or even use as a starting point for your semester project if desired.

The Resources tab will also contain instructions on how to install Node and Express JS as well as a tool that is useful for testing your application. Please use them if my instructions below are difficult to follow. If you have any issues or are confused by the content, please post in the discussion room or e-mail me directly with questions and optionally screenshots so I know where you are stuck exactly.

The Technology

npm

Node package manager (npm) is the, as it reads, package manager for the node.

Similar to pip installer for the Python applications and gem install for Ruby applications

There is a public collection of packages of open-source code for Node.js, front-end web apps, mobile apps, and other various JavaScript packages and npm allows us to access/download and install these packages locally on our machines

You can install packages globally and locally

npm install -g  // globally npm install  // locally

globally means you can access it from any directory or location on your computer

locally means you can only access the packages in the folder you run the command

NodeJS

An open source server framework

Runs on various platforms (Windows, Linux, Unix, Mac OS X, etc.)

Uses JavaScript on the server-sideThe benefits to having javascript on both server and client sides

No need to learn yet another language to add to the web application stack

Data can be in JSON and both sides will be able to easily parse it!

Very large community of developers so there's endless support on the web

NodeJS has the following qualitiesRuns single-threaded, non-blocking, asynchronously programming, which is very memory efficient

This means one thread handles all the client requests which then go to multi-threaded background workers handling the interactions with the DB and filesystem. It can accept a request and immediately be ready to handle another request.

ExpressJS

Express is a Node.js web application framework that provides features for web applications (web server)

It abstracts a lot of the details of web server development and hides it for you, letting you spend more time on the design and client-facing behaviors of your web application

Important Express application components:Node HTTP server - with this node module, we can tell our web server to listen on a specific port for HTTP requests (e.g. if our node HTTP server is running locally on port 3000, going to "localhost:3000" in the browser will be directed to our node HTTP server.

localhost is the host

3000 is the port

Router - the router will determine which request handler will handle the HTTP request. It matches the path of the URL (e.g. localhost:3000/this/is/the/path) to a request handler.

Request handler - a function that uses the request content, executes some business logic, and returns data of desired type as a response to the request (there will be an example coming up for clarity)

There are many more capabilities in Express that can help you develop a robust, powerful web application, but we will only be creating a simple HTTP web server this week. I encourage you to continue exploring other tutorials and learning how to expand upon the app we build today.

Web Servers

A web server is a software system that processes requests via HTTP, the basic network protocol used to distribute information on the World Wide Web. We will be building a web server with Node and Express JS this week.

Let's dive right in!

Note: the steps will require access to a console/terminal in a Windows/Mac/Linux OS. It is assumed you have access to a computer resource for this course by now so you should be able to accomplish all the tasks in this section. If you have trouble with any steps in this module, please post in the discussion room.

You can find screenshots and supplemental (better) instructions on this ExpressJS tutorial (Links to an external site.)Links to an external site.. Feel free to use the tutorial to speed along if you need, but the end goal will be to create a web service with HTTP routes return data.

Pick a location in your terminal and create a folder called 'CS80' with the following commands:

$ cd path/to/desired/install/location $ mkdir CS80

Go into the folder and initialize npm (this way you will be able to install node and express JS) with the following sequence of commands:

$ mkdir cs80 $ cd cs80 $ npm init

You will see a series of prompts from the npm installer, just press enter through all the questions until you are able see your terminal cursor again.

Now install Express as follows:

$ npm install express --save

You should now have a folder structure like the following:

cs80/ <-- this is your project directory node_modules <-- this folder holds all the installed package code package-lock.json <-- this is a file to lock the version of the packages you install with npm package.json <-- this is a file lists all packages and their versions

Install nodemon

nodemon will restart our application whenever we make changes to files in the directory

very convenient when developing as you can test your changes immediately

npm install -g nodemon <-- '-g' will allow you to run the command 'nodemon' anywhere

Create a file called app.js and paste this content into the file

var express = require('express'); var app = express(); app.get('/', function(req, res){ res.send("Hello world!"); }); app.listen(3000, function() { console.log("listening on port 3000"); });

Current file structure:

cs80/ node_modules package-lock.json package.json app.js <-- javascript file where your main program will run

Run your simple web server! Execute the following command in your terminal:

nodemon app.js

You should see something like:

$ nodemon app.js [nodemon] 1.11.0 [nodemon] to restart at any time, enter `rs` [nodemon] watching: *.* [nodemon] starting `node app.js` listening on port 3000

Open a browser of your choice and go to URL `http://localhost:3000`. You should see a plain web page with the words "Hello world!". If you do, congratulations, you've created a simple web server!

Breakdown

var express = require('express'); var app = express(); app.get('/', function(req, res){ res.send("Hello world!"); }); app.listen(3000, function() { console.log("listening on port 3000"); });

app.get(route, callback)

This function tells the application what to do when a GET request at the given route is received

The callback function has two parameters, request (req) and response (res)

request - represents the HTTP request and contains properties for the request query string, parameters, body, HTTP headers, etc.

response - represents the HTTP response that the Express app sends back when it receives an HTTP request

res.send(object)

This method takes an object and sends it back to the requesting client

You can pass JSON as well as stringsYou will need to set an HTTP header to let the client know what type of data is going back

e.g. res.set('Content-Type','application/json');

app.listen(port, [callback])

Defines which port to listen to for HTTP requests from a client and also defines a callback for when the listening has started

The example above prints to the console whenever the app starts listening on port 3000

Routing

We can use the app object to define different routes of your application. The app object includes get(), post(), put() and delete() methods to define routes for HTTP GET, POST, PUT and DELETE requests respectively.

GET - method to get data

POST - method to create data in a datastore

PUT - method to modify data in a datastore

DELETE - method to delete data in a datastore

Since we don't have an external datastore (MySQL database, SQLite database etc.) we will treat our application's memory as a datastore on the GET and POST calls for this module. The following example demonstrates configuring routes for HTTP GET and POST requests:

var express = require('express'); var app = express(); app.get('/book', function (req, res) { res.send('
HTML title Book!
'); }); app.get('/paper', function (req, res) { res.set('Content-type', 'application/json'); res.send({"paper":"this is JSON data about a cd"}); }); app.post('/pen', function (req, res) { res.send('Plain text data about pens'); }); app.listen(3000, function() { console.log("listening on port 3000"); }); 

You can send various types of data back to the client such as HTML, plain text, and JSON

The path specifies what comes after the "http://:" e.g. "http://localhost:3000/paper"

Accessing Request Data

ExpressJS does provide features to serve HTML/CSS/Javascript to the client browsers, but we will not be implementing that for the sake of this module. Please feel free to explore these framework capabilities on your own if you wish.

GET requests can have query parameters or filters for narrowing data search e.g. localhost:3000/book?title=chicken&author=john

The query params begin with a ? and take on a key=value format delimited by the &

You can put many query params, but some HTTP services will only accept a limited amount per URL

POST requests usually have data added to a HTTP request body. This body can hold various types of data and is generally limitless in size

You can access this request data from the 'req' object in the handler function

app.post('/pen', function (req, res) {...}

ExpressJS Example

Let's refine express application using both types of requests and turn it into a book data web server. You can find the complete example here (Links to an external site.)Links to an external site. as well.

// import packages we need to run the express app var express = require('express'); var app = express(); var bodyParser = require("body-parser"); // adding a body parser to handle JSON and url encoded form data for us automagically app.use(bodyParser.json()) app.use(bodyParser.urlencoded({ extended: false })); // our in-memory fake data store is just an array of javascript objects // we declare it here so that both endpoints can use it var books = [ { "author": "me", "title": "BookA", "pages": 600, "quality": "new" }, { "author": "you", "title": "BookB", "pages": 400, "quality": "used" }, { "author": "us", "title": "BookC", "pages": 500, "quality": "old" }, ] // request handler to search based on two fields: author and title app.get('/book', function (req, res) { // get the query params var title = req.query.title;

 // initialize the return data var data; // search for the book for (var i = 0; i < books.length; i++) { if (books[i].title == title) { data = books[i]; break; } } // pass JSON back to client res.set('Content-type', 'application/json'); res.status(201); // POST success response code res.send({"book": data}); });

// post handler to add a book to the hardcoded list // this is definitely not the correct model for storing data // this is simply an example app.post('/book', function (req, res) { // access the request POST body from the request object var data = req.body; // add the new book to the data store and return it var book = { "author": data.author, "title": data.title, "pages": data.pages, "quality": data.quality, } // add the book to the hardcoded list of books books.push(book) // return JSON list of books res.set('Content-type', 'application/json'); res.status(200); // GET success response code res.send({"books": books}); }); // listen for HTTP requests on port 3000 app.listen(3000, function() { console.log("listening on port 3000"); }); 

Observations

We declare our list of book objects before the HTTP routes, but THIS IS NOT GOOD PRACTICE. We are doing this for the sake of having a mock in-memory datastore we can use to test our endpoints. When we start learning about databases, you can modify the code and use an actual database to store books instead if you wish.

We add a body parser after importing our node packages. The body parser is a middleware that helps parse JSON and form data.middleware is any number of functions that are invoked by the Express routing layer before the request handler is. It sits in the middle between a raw HTTP request and the final intended route.

Used for handling content-type, logging, and much more.

We now have a HTTP GET route /book that accept query params. These params specify certain search fields we want when looking for data. In the example above you would make the request with the author and title query params to find the book object which will be returned back to the client.

Try inputting `http://localhost:3000/book?title=BookA` (Links to an external site.)Links to an external site. into the browser URL bar and you should see JSON formatted data for the book return.

The POST request takes a JSON body that contains the book javascript object that we want to store and adds the incoming book data into our books list, and finally returns all books including the new book to the client in JSON format.

The response content-type is set to "application/json" to let the the client know to accept the web server's response as JSON content-type

The response status is a HTTP status code set based on the incoming method of the HTTP request:

200 - GET, PUT - represents a success and typically includes response body

204 - DELETE - represents a success and typically excludes the response body

201 - POST - represents a success and sometimes includes a response body

Application Programming Interface

Also known as API for short

An API is a collection of endpoints (or HTTP routes that are accessible to an outside user or client)

The API is the interface in which your client interacts with a web server

APIs provide entrypoints to the system that can be defined and tailored to the needs of the client and require the HTTP protocol.

How do we test an API?We test with tools/applications that allow us to make HTTP requests outside the browser, such as Postman (Links to an external site.)Links to an external site.. We can also do this from the command line interface. Refer to the Resourcestab for how to install cURL on various operating systems.cURL is a command line tool for getting or sending files using URL syntaxcurl "localhost:3000/book?title=BookA"

GET request via cURL

?key=value is our query parameters block that specifies what data to get.

curl "localhost:3000/book" -d '{"author":"them","title":"divergent","pages":550, "quality":"new"}' -H 'Content-Type:application/json'

POST request via cURL

'-d' is the HTTP request body in JSON format

'-H' is is a flag for adding headers to the request, telling our web server what type of data is coming

Congratulations! You just created your own web server with a public-facing API. Try using cURL to test your new web server and modify however you want. In the coming chapters, we will learn how to bring the client-side and server-side together to form a full stack web application: front-end interface web page (HTML/CSS/Javascript), a back-end web server (ExpressJS), and a database (MySQL, etc.). Be sure to keep your work in this chapter handle as we will be re-using the code later.

The Model-View-Controller (MVC) Architecture

Modern web apps almost always follow this architectural design pattern, so it is very important to be familiar with it. ExpressJS does not inherently follow the MVC architecture but there are simple structural changes you can do to make your app follow an MVC architecture. Learn more about the architecture here (Links to an external site.)Links to an external site..

MVC consists of three separations of concerns:

Model - handles the state of the data in the application (database models)

View - handles the presentation of the data (what the user sees)

Controller - accepts the incoming events from a client and operates on the model and views

If we translate this to ExpressJS speak:

Model - This is a javascript file that initiates the conversation between the web app and the database and provides helper methods for operating against the database itself (Create, Read, Update, Delete).

View - This is where we store all our public facing files such as HTML, CSS or templates to be accessed by the controller when data is ready to be presented to the client requesting information.

We did not implement this in our web server this time around, but feel free to try it out using any of the tutorial websites I posted in the Resource tab

Controller - This is where the routing and business logic happens.

When a request comes in the controller routes it to the right business logic, that business logic could operating on the database (e.g. creating a user entry in the database), once the response data is ready, the controller will operate on the view to apply the data (e.g. populate an HTML element) and send it back to the client all ready to be displayed.

Problem 1

Now that we've seen a web server in action. Please extend the existing implementation from the Lecture Notes and add the following routes in your ExpressJS web application:

1. DELETE /book?title=

returns 204 Not Content status code

no response body

2. GET /books

returns 200 Success status code

returns list of all books

return format must be JSON:

{ "books": [{ "author":"joe", "title":"MyBook", "pages":234, "quality":"new"}, ... ] }

3. PUT /book

returns 200 Success status code

updates a single book in the list

returns a single book formatted in JSON:

{ "book": { "author":"joe", "title":"MyBook", "pages":234, "quality":"new" }}

I should be able to use cURL to test your existing AND new routes. Make sure they all work as expected.