The Serverless Testing Guide provides greats concepts to follow, like:

• Write your business logic so that it is separate from your FaaS provider (e.g., AWS Lambda), to keep it provider-independent, reusable and more easily testable.
• When your business logic is written separately from the FaaS provider, you can write traditional Unit Tests to ensure it is working properly.

So, how do we follow those concepts?

Testing starts with understanding the Pyramid. The layers are Acceptance, Integration and Unit. Ideally you want to focus more on the Unit and Integration tests. In this tutorial, we will focus on setting up a project that can test the units and integrations.

Project Structure

• We will start with the Serverless Framework configure for NodeJS and Webpack. If you need help on setup then checkout this tutorial.
• We will code by separating out business and provider logic from the services and unit libraries.
• We will leverage Jest. Jest focuses on simplicity and makes it easy to add code coverage, mocks and snapshots assertions.

Step 1

• Install Jest. Detailed instructions here

yarn add --dev jest

Step 2

• Configure Jest jest.config.js
• In this configuration we will setup the directories we want to be including in code coverage. Visit the docs for details on coverage thresholds
• Also, I like using aliases with Webpack to utilize absolute instead of relative paths. Jest defines this as moduleNameMapper and uses babel under the hood.

// jest.config.js
module.exports = {

  collectCoverageFrom: ['./app/services/**/*.js'],

  coverageThreshold: {
    './app/services/**/*.js': {
      branches: 100,
      functions: 100,
      lines: 100,
      statements: 100,
    },
  },

  moduleNameMapper: {
    '@app(.*)$': '<rootDir>/app/$1',
  },
}

Step 3

• Lets write some code for testing

// app/services/sum.js 

export default (a, b) => {
  return a + b;
};

export const random = () => {
  return Math.random();
};

Step 4

• Lets write the Jest test
• Note the @app from syntax, that is the moduleNameMapper mapping to <project directory>/app/services/sum

import sum from '@app/services/sum';

test('adds 1 + 2 to equal 3', () => {
  expect(sum(1, 2)).toBe(3);
});

Step 5

• Run the test with coverage

jest --collectCoverage

This setup allows us to separate out concerns.

• Controllers should implement provider specific logic and source the business logic to the services or libraries.
• Services code should have unit or integration tests with Jest.
• Development cycles should improve because developing in units will have quicker feedback cycles.

Conclusion

In this tutorial we went over setting up Jest to test the units and integrations parts of the Test Pyramid. In the next few weeks, I'll go over the approach on the last piece of the Pyramid, which is running acceptance tests.

If you would like to see a working example you can check out this directory in my node-learning repository.

Serverless Framework with AWS Provider helps you develop and deploy your AWS Lambda functions, along with the AWS infrastructure resources they require.

Webpack

Webpack is a static module bundler for modern JavaScript applications.

Benefits

• Compiler Configuration
• Module Aliases
• Loaders
• Next Generation JavaScript with Babel Loader
• Immutable builds

Setup

1. Create a new service with Serverless Framework.
2. Install Serverless Offline and Serverless Webpack
3. Configure the serverless.yml

service: controllers # NOTE: update this with your service name

plugins:
  - serverless-webpack
  - serverless-offline

provider:
  name: aws
  runtime: nodejs8.10
  stage: master
  region: us-west-2

Configure Webpack

Leverage serverless-webpack to handle the entrypoints. This is beneficial as you only have to configure the function handlers in the serverless.yml file.

const slsw = require('serverless-webpack');
// more requires… 

module.exports = {
  entry: {
    ...slsw.lib.entries,
  },
// more webpack config… 
};

Setup the function endpoints

In the serverless.yml file add

// previous serverless.yml 
functions:
  users:
    handler: app/controllers/users.default
    events:
      - http:
          path: api/users/{proxy+}
          method: any

Note: If you disapprove of the Services Pattern and would rather opt for Microservice Pattern then you can implement each http method in separate functions and specifying the http method in the events.

Add the Code

In app/controllers directory you can add the function logic. I typically separate validating and normalizing the data in the controller and pass that down to a model. I can write more on this in another article.

//app/controllers/users.js

class UsersController {
  async handle(event, context) {
    try {
      const method = event.httpMethod.toLowerCase();
      return await this[method](event, context);
    } catch (e) {
      console.error(e);
      return {
        statusCode: 500,
        body: JSON.stringify({ error: true }),
      };
    }
  }

  async get(/*event , context*/) {
    return {
      statusCode: 200,
      body: JSON.stringify({
        users: [], // retrieve from data source
      }),
    };
  }

  async post(/*event , context*/) {
    return {
      statusCode: 200,
      body: JSON.stringify({
        users: {}, // create in data source
      }),
    };
  }

  // continue similar implementation for update and delete...

}

export default async (event, context) => {
  return await new UsersController().handle(event, context);
};

Note: You can setup the directories and file name to whatever you like just make sure to update the serverless.yml handler

Develop Offline

By installing serverless-offline it allows you test out the code locally

$ serverless offline start --dontPrintOutput

Deploying

Serverless Webpack bundles the project for serverless. If you want to include/exclude specific node packages you can do that in the serverless.yml configuration for serverless-webpack.

$ serverless deploy

Conclusion

In this tutorial you learned how to:

• Implement Services Pattern
• Configure Serverless Framework
• Configure Webpack
• Develop Offline
• Deploy

If you would like to see a working example you can check out this directory in my node-learning repository.

Instead of having to think about how to provisioning and automating deployments for each business opportunity, you can follow a set of opinions laid out by the service provider. This allow for great third party tools like Service Framework.

The initial cost is one of the most attractive reasons to utilize Serverless. The free tiers and pay as you go model works well when trying to start an idea. Instead of having to pay upfront you can launch new business initiatives and only pay when there is product market fit.

Today, I only deploy new ideas with Serverless, because of:

• Cost
• Business Focus
• Sleep

Great Los Angeles - AWS Users Group event last night recapping the AWS re:Invent 2018.

Randall Hunt definitely knows his AWS and covered variety of topics like:

• AWS Business Growth & Competition
• Global Infrastructure
• Compute, Containers, Serverless
• Storage & Analytics
• Databases
• AI ML
• Security
• IOT

These are what I'm excited to test out:

• AWS Global Accelerator
• ALB support for Lambda
• Lambda Layers
• API Gateway Websockets
• AWS Cloud Map
• AWS Step Functions
• Deep Racer

Storybook is the UI Development Environment that I've been looking for. You can:

• Visualize
• Document
• Manually Test
• Automate Visual Test
• Produce a Static output of the Storybook that can be hosted/shared
• Plus much more with these addons or you can create your own

The Storybook devs have put in a good amount of effort but there were some issues on getting it working in Emberjs. I hope with this post, I can share my setup so we can make it easier in the future.

Notable links that helped me setup the project:

• https://storybook.js.org/basics/guide-ember/
• https://github.com/storybooks/ember-cli-storybook/pull/3/files
• https://github.com/storybooks/storybook/tree/next/examples/ember-cli

Example Project

• Demo
• Implementation

Steps

• Note yarn can be switched for npm run
• Yarn scripts can be found in the package.json

1) Install the dependencies and add the scripts

2) Setup the Storybook config files

3) Write some stories

4) Run Ember and Run Storyboard

• $ yarn start
• $ yarn storybook
• NOTE: had issues making this one command so thats why it they are separate commands that should be ran in different terminals.

5) Configure Jest & write some jest image snapshot tests

• $ yarn jest:integration

6) Create a share able Storybook distribution

• $ yarn storybook:dist

Addons

• Knobs lets you create color pickers, dropdowns and ability to change text. Great for manual testing during development.
• Notes Allows you to document the component
• Viewport allows you to visualize the look on different screens.

Visual Testing

• Jest allows you to do image snapshot setting for visual regression test. Click to see demo.

Still new to Storybook and know this wasn't an in depth tutorial so tweet at me if you need help or want to see more.

Meetup hosted by Product School

Rory Joyce started as a Product Manager (PM) at the company he founded. He is now at Credit Karma as the director of product collaborating with many other PMs. He discussed his experience in the many areas of PM. Theses areas include:

• Product Strategy
• Experimentation
• User Research
• Communication
• Partnership

Product Strategy starts off as a drunken walk to market fit, then hyper growth and scaling. Theses cycles of PM are interesting as the can happen again with in a cycle. You can be scaling one product and launching another having to start the drunken walk all over again to find market fit.

During the iteration, you will always feel product market fit when it happens. The strategy requires identifying the user problem and crafting a solution that drives business. This process start with a thesis that needs to get validated. Key performance indicators (KPI) are set to measure success that allows for understanding market fit. KPIs can be metrics that tie to revenue. Using these metrics to iterate and validate can lead to market fit.

Experimentation help explore concepts and validate features. You need to try many different things that should be on opposite spectrums. For example, if you test UI/UX changes you need to go 3 times difference. Don’t just change the color of a button.

User research requires getting outside and forming your point of view. Interviews, surveys and usability studies will all contribute to understanding the user.

Communication is the most import skill for a PM. You need to be clear consistent and compelling.

PMs must build trust with peers and foster productive Partnerships with engineers, UI/UX, marketers, analysts and stakeholders. The PM must be optimistic, show empathy and be open minded.

Figure out what energizes you because PM has many ups and downs so you better work on something you love doing.

Thank you Rory Joyce on sharing your experiences and the Product School for organizing the meetup.

<body>
  <div class="master-detail">
    <div class="master">
      <div class="scroll">
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
        <div class="item">Item</div>
      </div>
    </div>
    <div class="detail">
      <div class="scroll">
        <p>
          Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed nec ullamcorper turpis, nec posuere nisi. Mauris imperdiet, urna non porta vestibulum, nisl libero dapibus purus, nec egestas quam risus placerat nisl. Donec a mi eget tortor ultrices convallis.
          Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Mauris vestibulum auctor mi quis consequat. Curabitur imperdiet porttitor leo, vel consequat nisl iaculis eget. Donec porta sodales lobortis. Mauris ullamcorper,
          sapien et blandit viverra, erat sapien ullamcorper dolor, ut feugiat nulla odio vehicula sapien. Ut lobortis, lacus eget cursus tincidunt, lectus augue vehicula neque, eget cursus lorem neque in ipsum. Proin dignissim condimentum quam, eu pharetra
          risus faucibus at.
        </p>
        <p>
          Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed nec ullamcorper turpis, nec posuere nisi. Mauris imperdiet, urna non porta vestibulum, nisl libero dapibus purus, nec egestas quam risus placerat nisl. Donec a mi eget tortor ultrices convallis.
          Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Mauris vestibulum auctor mi quis consequat. Curabitur imperdiet porttitor leo, vel consequat nisl iaculis eget. Donec porta sodales lobortis. Mauris ullamcorper,
          sapien et blandit viverra, erat sapien ullamcorper dolor, ut feugiat nulla odio vehicula sapien. Ut lobortis, lacus eget cursus tincidunt, lectus augue vehicula neque, eget cursus lorem neque in ipsum. Proin dignissim condimentum quam, eu pharetra
          risus faucibus at.
        </p>
        <p>
          Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed nec ullamcorper turpis, nec posuere nisi. Mauris imperdiet, urna non porta vestibulum, nisl libero dapibus purus, nec egestas quam risus placerat nisl. Donec a mi eget tortor ultrices convallis.
          Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Mauris vestibulum auctor mi quis consequat. Curabitur imperdiet porttitor leo, vel consequat nisl iaculis eget. Donec porta sodales lobortis. Mauris ullamcorper,
          sapien et blandit viverra, erat sapien ullamcorper dolor, ut feugiat nulla odio vehicula sapien. Ut lobortis, lacus eget cursus tincidunt, lectus augue vehicula neque, eget cursus lorem neque in ipsum. Proin dignissim condimentum quam, eu pharetra
          risus faucibus at.
        </p>
      </div>
    </div>
  </div>
</body>

html,
body {
  height: 100%;
  margin: 0;
  font-family: Arial;
}

.master-detail {
  display: flex;
  flex-direction: row;
  height: 100%;
  flex: 1 1 auto;
}

.master-detail > .master,
.master-detail > .detail {
  display: flex;
  flex-direction: column;
  overflow-y: scroll;
  overflow-x: hidden;
}

.master-detail > .master {
  background: #EAEAEA;
  border: 1px solid #CCC;
  flex: 1;
}

.master-detail > .detail {
  background: #f0f0f0;
  flex: 2;
}

.scroll {
  flex: 1 1 auto;
  overflow: auto;
  min-height: 0px;
  height: 100%;
  padding: 10px;
}

.item {
  background: white;
  border-radius: 5px;
  margin: 5px;
  padding: 20px;
}

https://jsfiddle.net/adinoblur/qhpko0uo/4/

Consuming JSON APIs can be challenging. jq is a light weight command line tool to parse json. It can solve problems like totaling a json attribute so you don't have to do it manually.

Goal

Sum the total attribute in each requestReport object.

example data set

{
    "requestReports": [
        {
            "id": "1",
            "total": "1"
        }, 
        {
            "id": "2",
            "total": "3"
        },
        {
            "id": "3",
            "total": "5"
        }
    ]
}

jq reduce

jq '.requestReports | reduce .[] as $item (0; . + ($item.total|tonumber)) '

jq play ground example

https://jqplay.org/s/JJ2fqpEHib

References

• tonumber to convert string to number https://stedolan.github.io/jq/manual/#tonumber
• reduce to add up the total attr https://stedolan.github.io/jq/manual/#Advancedfeatures > Reduce

• Official Node container to control node versions between projects.
• Ember CLI container to start and work on ember projects.

1. Create a docker folder in your monolithic application. The Dockerfile and file configuration will be stored there. You may want a folder per container. Extend from the official docker containers. There are requirements on running containers the "Docker Way," so using the official containers will ensure that. https://hub.docker.com/explore/

```bash |-- app | |-- controllers | |-- models | |-- views |-- docker | |-- nginx | |-- Dockerfile | |-- php | |-- Dockerfile | |-- mysql | |-- Dockerfile

2. Do it the docker way. Create a container for each purpose This means a container for the code, web server and persistence layer. 

- [Nginx](https://hub.docker.com/_/nginx/)
- [PHP](https://hub.docker.com/_/php/)
- [MySQL](https://hub.docker.com/_/mysql/)

3. Use [Docker Compose](https://docs.docker.com/compose/overview/). Starting the environment will be as easy as running: 

bash docker-compose up -d ```

4. Iterate on docker containers as if you were iterating on code. Create a feature branch and merge only when its ready for use.

Continuous Integration

1. Create a specific container to run the tests. For example, you can use a phpunit container. This encapsulates the testing environment in a single container. Thus any developer or machine can run the tests the same way.

2. Automate tests. I prefer Shippable since it embraces docker, use your own machines and affordable. Theres many options out there so definitely choose one that works with you and your team.

Production

Once you've enabled containers for development and continuous integration, it will be time to release into production. The next step is to research the various docker schedulers and infrastructures.

• AWS ECS
• Kubernetes
• Mesos
• Docker Swarm Mode

Currently, I have chosen AWS ECS since it abstracts most of the cluster configuration. Its as easy as launching an AWS EC2 instance into the ECS cluster and configuring an ECS Service. The ECS service is defined by the task definition which is basically a json representation of the docker-compose file. Also, placing an AWS ELB in front of the ECS service provides managed load balancing and can enable rolling updates. Lastly, AWS Autoscaling will enable EC2 instance scaling and rotation in case a server goes down.

The important goal when shipping docker containers into production is immutability. During the development you will most likely mount your code into the container. In production, we want to build the container with the code and ship the immutable container. This will ensure the code's state. If there's an issue, the container can easily get pulled locally and debugged.

Continuous Delivery

The last step is to automate the delivery. This is pretty difficult and depends on infrastructure provider and scheduler, so I wont go into too much detail on this post.

I use Shippable + Terraform to deploy the AWS ECS service. I update the task definition file to point to the new artifact tag stored in AWS ECR. Then run Terraform apply to deploy the new ECS Service. This means once a pull request gets merged, shippable deliveries the code through automated terraform apply.

Summary

Containerizing the monolithic app enables quicker development cycles, automated continuous integration and deployment. These cycles didn't happen all at once. It started slowly but overtime Docker has enabled complete automation from development to production.

• Docker can ease pain with continuous integration and delivery.

Future Posts

• Getting started may seem daunting but you can start with the development environment then move towards production.
• Docker puts environment into code. Terraform puts infrastructure into code.

Learn More

• Docker
• Docker Build
• What is a container?

Best way to get in contact with me is to tweet me @mrazdean.