docs/quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import uuid\n",
    "\n",
    "pipeline_dir = os.path.join(os.getcwd(), '..')\n",
    "if pipeline_dir not in sys.path:\n",
    "    sys.path.append(pipeline_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pipeline import Pipeline, events, resources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started\n",
    "You can create a new project with the `pipeline-create <new_directory>` which generates a template project in a new directory.  Edit the `handler.py` file to build your pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a Pipeline\n",
    "Pipelines are made up of lambda functions, events, resources, and services.  Let us start by defining an empty pipeline by inheriting from the base class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyFirstPipeline(Pipeline):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super().__init__()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define a Function\n",
    "We can define a lambda function by defining a class method decorated with an event trigger.  The decorator determines how the lambda function is executed.  Let's write a simple \"Hello World\" function triggered with a GET request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyFirstPipeline(Pipeline):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "    \n",
    "    @events.http(method=\"get\", path=\"helloworld\", cors=\"true\")\n",
    "    def hello_world(self, event, context):\n",
    "        print(\"Hello World!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define a Resource\n",
    "All that is required to build a working pipeline is a `pipeline.Pipeline` object with at least one lambda function decorated by an event trigger.  If we want our pipeline to interact with other AWS resources we may define a resource.  Let's define a DynamoDB table which we will use to store messages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyTable(resources.DynamoDB):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.add_attribute('id', 'S')\n",
    "        self.add_key('id', 'HASH')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also edit our original lambda function to take user input in the GET request.  We can register our newly created resource by instantiating and passing into the call to `super`.  Additionally, resources provide helper methods for interacting with the resource.  The DynamoDB resource, for example, inherits the `put`, `get`, `delete`, and `list` methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "table = MyTable()\n",
    "\n",
    "class MyFirstPipeline(Pipeline):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super().__init__(resources=[table])\n",
    "    \n",
    "    @events.http(method=\"get\", path=\"helloworld/{message}\", cors=\"true\")\n",
    "    def hello_world(self, event, context):\n",
    "        item = {'id': str(uuid.uuid1),\n",
    "                'message': event['message']\n",
    "               }\n",
    "        # Use helper method\n",
    "        table.put(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Packaging\n",
    "Now that your pipeline is built, we need to do some final packaging before we can deploy.  Because lambda can't call functions inside of classes, we need to expose each decorated method.  Additionally, we need to add a deploy wrapper for deploying the pipeline.  Once finished your project should look something like `cognition-pipeline/examples/quickstart/`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate your pipeline\n",
    "pipeline = MyFirstPipeline()\n",
    "\n",
    "# Lambda func entrypoint\n",
    "hello_world = pipeline.hello_world\n",
    "\n",
    "# Deploy wrapper\n",
    "def deploy():\n",
    "    pipeline.deploy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements\n",
    "Requirements may either be added to `requirements.txt` in the pipeline directory or handled through the use of services."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploying\n",
    "Once your pipeline is packaged it is time to deploy!  Use the `pipeline-deploy <pipeline_directory>` to generate a `serverless.yml` file which may be deployed with Serverless Framework."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing\n",
    "Cognition-pipeline integrates with unittests by providing an object-oriented framework for interacting with your functions and resources.  Simply pass your instantiated pipeline into the standard `unittest.TestCase.setUp` method and test your functions!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import unittest\n",
    "import time\n",
    "\n",
    "class MyFirstPipelineTestCases(unittest.TestCase):\n",
    "\n",
    "    def setUp(self):\n",
    "        self.pipeline = pipeline\n",
    "\n",
    "    def test_hello_world(self):\n",
    "        func = self.pipeline.functions['hello_world']\n",
    "        table = self.pipeline.resources['QuickstartTable']\n",
    "        # Invoke your lambda function (creates a new entry in DynamoDB Table)\n",
    "        func.invoke('test_entry')\n",
    "        # Give the table some time to update\n",
    "        time.sleep(5)\n",
    "        # Confirm the new entry using a DynamoDB table scan\n",
    "        response = table.list()\n",
    "        self.assertEqual(len(response), 1)\n",
    "        self.assertEqual(response[0]['message'], 'test_entry')\n",
    "        # Delete the new entry with the entry's id\n",
    "        table.delete(response[0]['id'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}