An Edge Machine Learning Systems Design Example

Edge computing is intended to bridge the gap between cloud infrastructure and devices at the network edge by bringing computing capabilities to network endpoints. By provisioning computing power closer to end-user equipment or embedded devices, users can implement edge machine learning with a dedicated computing resource that does not require access to cloud computing resources. As much as the industry has created hype around these systems, we’ve done a poor job of making these tangible for electronics systems designers.

To help designers get more familiar with this area of computing and machine, and to illustrate what is possible in these systems, we’ll outline a design example for an edge machine learning system in this article. This design example shows the system architecture that is suitable for an edge machine learning server and the types of components users should expect to see in these systems. From this example, designers can proceed further to add more features and determine a form factor that will meet their deployment requirements.

What’s Needed in an Edge Computing System

Before getting started on a design example that could be used to implement machine learning at the edge, we need to understand some of the functional requirements for these systems. This type of system could be deployed at a base station or in user facilities (e.g., industrial environment), or built into infrastructure. The list of requirements below is not comprehensive, and it illustrates the functional considerations that go into edge computing systems design.

Processors

Modern edge machine learning systems contain multiple dedicated processing cores precisely trimmed and optimized for specific tasks. These are delineated above as the main processor, which runs applications and controls data flow, and the ML processor, which implements training and/or inference. This is in contrast to an embedded device at the edge, where the ML core might be implemented on the same die or package as the main CPU.

Memory

One of the main challenges in edge computing systems design is memory bandwidth requirements. Memory needs to have a sufficiently high transfer rate to ensure input or training data can be processed in a reasonable amount of time. The role of memory in this case is to store input data or training that is being passed from the CPU to the ML processor. It is possible for the ML processor to have its own dedicated set of memory modules so that AI compute workloads do not interfere with other processing tasks. GDDR is one option for the ML processor, while standard DDR is acceptable for the main processor.

An Example Architecture

An example architecture for an edge machine learning system is shown below. In this system, there is no assumption as to how the processed data is being collected by the system. We can assume that a data tagging process is implemented on the edge server or by the endpoint device at the time of data collection. It is assumed that data is passed to the main and ML processors in digital form, where it is then used for training and/or inference.

There are some important points to note about this architecture:

1. The CPU is placed on the motherboard directly, but it could be placed as a module. Modules are beneficial because they offer some future-proofing.
2. Components on the expansion cards could also be mounted directly on the main PCB if desired.
3. The RFFE (RF front-end) provides any required wireless capabilities in the design so that an external network or device could be accessed. This could be through WLAN or cellular, although some systems might use Sub-GHz radio links.
4. The memory sub-section is shown as being split among the main processor and the ML processor. This is because embedded software applications and the ML processing need different levels of memory access.

Beyond System Architecture

Apart from the system architecture requirements listed above, designers must build a real system that can survive in the system’s envisioned deployment environment. This puts the focus on implementing the above architecture into a physical design. The individual systems should be deployed on PCBs and packaged in a rugged enclosure. Some of the physical design points include:

• Motherboard and expansion card designs may be on separate PCBs
• External connectors should have appropriate IP rating
• Enclosures should ideally be shielded so they can withstand RFI and ESD
• A cooling strategy is needed, which requires some airflow to the ML processor

A set of systems-level simulations can be useful to better understand cooling challenges, identify areas of a PCB creating excessive EMI, and for simulating individual interconnects. S-parameter extraction, thermal demands, broadband emissions, and mechanical reliability are all challenges in embedded systems, but they are particularly important in edge computing systems that must be highly reliable.

Anything from simple embedded devices to advanced edge machine learning systems can be designed and evaluated with the complete set of system analysis tools from Cadence. Only Cadence offers a comprehensive set of circuit, IC, and PCB design tools for any application and any level of complexity. To learn more about interconnect simulation in high-speed systems, watch our webinar, Understanding Simulation Analysis Parameters for Serial Link Systems in SystemSI.

Subscribe to our newsletter for the latest updates. If you’re looking to learn more about how Cadence has the solution for you, talk to our team of experts.