04-652   Artificial Intelligence System Design

Course discipline

MS EAI

Course description

Artificial Intelligence can be defined as the study of the design of rational agents where an agent is something that acts, and rational means that the agent acts to achieve the best-expected outcome. AI has a long history, extending at least as far back as the 1950s. Machine learning can be considered as a sub-field of AI focused on how a rational agent can adapt to new circumstances as well as detect and extrapolate patterns. The implementation of an AI system typically relies on many different hardware and software components.

This course focuses on two aspects of AI system design:

• Performing ML inferences in resource-constrained edge devices such as one might find in an IoT network.
• ML model performance analysis and maintenance in the cloud. Students will study Edge ML in the context of Raspberry Pi's connected to Arduinos, and use TensorFlow to design models. They will use various AWS and web technologies to learn how to design distributed and cloud-based systems for analyzing and maintaining ML models. This course provides a foundation students can use to design and implement complex AI systems that use both edge and cloud components.

Learning objectives

Students will learn how the edge components of an IoT-like network can be connected to edge gateways, and how the data aggregated there can be further routed to the cloud. They will study important edge system design tradeoffs, such as resource consumption (e.g., memory and power) vs. ML inference speed. They will learn the importance of matching ML models to edge device constraints. Students will also learn how data can be collected in devices outside the cloud and then routed to the cloud for further analysis. They will learn the importance of using designs such as REST APIs. Finally, they will learn the fundamentals of how to assess model performance automatically in the cloud and detect when model updates should be triggered.

Outcomes

After completing this course, students will be able to do the following:

• Configure a Raspberry Pi as an edge gateway
• Use the Raspberry Pi as a development platform for designing simple TensorFlow ML models
• Develop programs incorporating TinyML models for Arduino's on the Raspberry Pi, and deploy them
• Communicate data bi-directionally between the Raspberry Pi and Arduino in real-time, and assess the real-time performance of edge ML models running on the Arduino, including memory utilization and speed of execution
• Understand principles of REST API design and complete client and server-side implementations for accessing ML models using common SDKs such as Flask for Python applications.
• Understand principles of cloud-based identity and access management such as roles, policies, users, groups, and API key credentials
• Stream data to a cloud database, such as DynamoDB (AWS), where it can be analyzed and queried
• Analyze data stored in the cloud to assess model performance
• Update ML models

Content details

• Introduction to distributed ML systems such as IoT systems using edge ML
• Configuring Raspberry Pi gateway devices running Linux
• Introduction to ML with an emphasis on neural networks and CNNs
• Writing python code to execute on RPi's, and C code for Arduinos
• Using TensorFlow to design EdgeML models
• Assessing the performance of TinyML models w.r.t memory and execution speed
• Review of model quality metrics including accuracy, precision, and recall; understand how data distributions can lead to misleading quality metrics (e.g., under conditions of class imbalance)
• Overview of web platform architectures: TCP/IP, DNS, URI/URL/URNs
• Principles of web resources behind the REST API framework
• Identity and Access Management (IAM) in Amazon web services as an example of a common IAM framework.
  SDKs for streaming data to AWS (e.g., DynamoDB) and monitoring using serverless (Lambda) functions
  Detecting model discrepancies by monitoring quality (e.g., class confidence) and retraining when needed

Prerequisites

None

Faculty

Mike Perkins and Charles Wiecha