ArcGIS in Lambda

Introduction

Running the Python API for ArcGIS in AWS Lambda can unlock powerful capabilities for automating tasks, managing your ArcGIS Online Organization or Portal Installation, or creating slick server-less dashboards.

If you are not familiar with AWS Lambda, you can find more information here: https://aws.amazon.com/lambda/.

Package the Python API for ArcGIS

In order to host the Python API for ArcGIS scripts within AWS Lambda, you will need to first package the API and its dependencies using the same OS as Lambda. One easy way to do this is to spin up a new EC2 instance for packaging.

  1. Create an EC2 Instance with the Amazon Linux Flavor and all free tier defaults.
  2. Connect to your instance using Putty or your favorite SSH Client.
  3. Download and install Anaconda:

    curl -O https://repo.anaconda.com/archive/Anaconda3-5.3.1-Linux-x86_64.sh
    bash Anaconda3-5.3.1-Linux-x86_64.sh
    
  4. Install the Python API for ArcGIS:

    conda install -c esri arcgis
    
  5. Install a seperate instance of Python (we will use this to set up a virtual environment for a smaller deployment package):

    sudo yum install python3
    
  6. Install virtual-environment:

    python3 -m pip install --user virtualenv
    
  7. Create a new virtual environment:

    python3 -m virtualenv arcgis-lambda
    
  8. Copy the Python API for ArcGIS into your new environment:

    cp ./anaconda3/pkgs/arcgis-1.5.3-py37h39e3cac_1/lib/python3.7/site-packages/arcgis/ ./arcgis-lambda/lib/python3.7/site-packages/ -r
    
  9. Activate your new virtual environment, install dependencies, and deactivate:

    source ./arcgis-lambda/bin/activate
    pip3 install numpy
    pip3 install pandas
    deactivate
    
  10. Navigate to your virtual environment and create a zip archive:

    zip -r ../../../../arcgisLambda.zip .
    
  11. Use Filezilla or your favorite SFTP Client to download your newly created archive.

Now that everything has been packaged up, you can feel free to stop your EC2 instance. You may want to keep it around in case you want to add new dependencies down the road.

Add Functionality

  1. Use Filezilla or your favorite SFTP Client to download your newly created archive.
  2. Add an AWS Lambda compatible Python file to the root of your zip archive. For Example:

    from arcgis.gis import GIS
    
    def getVirginiaContent(event, context):
      gis = GIS()
      content = gis.content.search("Virginia")
      return {
        "Virginia": content
      } 
    
  3. Create a new Python 3.6 Lambda function in AWS. The handler should be fileName.functionName.

  4. Test it out!

Conclusion

Once you have a working function, you can add encrypted logon information as environment variables, hook up triggers, and otherwise integrate into a larger server-less architecture.

Leave a comment below letting me know what you’ve been able to create or if you’re having any issues.

Good Luck!


comments powered by Disqus