Introduction to MTF

MTF overview

Modular Test Framework (MTF) is a testing framework designed for creating, executing and validating tests.
It consists of a set of tools that work together to provide the necessary functionalities for various tasks.
These tools can be combined to meet specific requirements.
When coupled with our hardware modules (CMs, BTSRevo, EES, etc.), MTF can function as a Hardware-in-the-Loop (HIL)
system, which is is a real-time simulation setup used primarily for testing and developing complex embedded systems,
especially in automotive applications.
Hence, we refer to these tools as the MTF Toolchain which will be explained in the following sections.

MTF Toolchain features

MTF Toolchain enables users to:

  • Access the vehicle communication technologies (CAN, CAN FD, LIN, FlexRay, Ethernet).

  • Execute diagnostics and perform ECU measurement/calibration.

  • Write residual bus simulations as well as complex and time-critical test functions.

  • Create and manage test cases.

  • Execute test cases in a HIL System.

  • Generate logs and traces.

  • Post-process and analyze test cases traces.

  • Online visualization of Frame signals, I/Os measurement values, etc.

MTF Toolchain modules

MTF Toolchain includes:

Libraries for:

  • Packet crafting and decoding: ANDi SDK.

  • Communication Matrix Parsing (FIBEX, ARXML, DBC, LDF, etc.): DataBex

  • Technica devices discovery and configuration: automated_hardware_configurator

  • Building and interpreting diagnostics jobs: DiagToolBox

Test Automation Framework
Test Development Environment
HIL and Simulation platform

  • MTF Core

MTF Analyser: Trace Analysis Tools

  • MTF UI (ANDi) for offline signal plotting and Wireshark Profiles generation

MTF components

As previously mentioned, the MTF (Modular Test Framework) toolchain consists of multiple aligned tools. For instance, they share the same core while providing users with various solutions or perspectives. In this section, we will dive into the different components of MTF.

You can find a brief summary of the tools available in MTF in the image below

Starting at the base we observe:

  • The hardware devices the user wants to use.

  • The shared core connecting the hardware world with the user-level world.

  • The APIs providing the core functionalities to any app that requires them.

  • The App layer that offers the user the desired functionalities.

We will explore them in depth in the following sections.

MTF.core

The mtf.core is the foundation of the MTF Toolchain. All tools depend on this library as it contains the primary business logic. It establishes a Hardware Abstraction Layer of the supported devices, ensuring consistent interfaces in the API layer for necessary operations. It also includes logging and simulation capabilities, understands multiple protocols (CAN, LIN, FlexRay, SOME/IP…) and provides unified access to any OEM required database of signals and messages.

Developed in C++, it offers:

  • Full support for multiple operating systems: Windows and Linux.

  • Fast speed, thanks to low-level integration.

  • The ability to expose the functionalities to other languages through APIs.

APIs

API is the acronym for Application Programming Interface. It enables two software components to communicate with each other using a set of definitions and protocols.

In order to communicate with the mtf.core, its functionalities are exposed to other components, or even directly to any interested user, through APIs. The main API is Python-API. It allows the commuication with mtf.core functions through Pyhton language, enabling users to create test cases (either directly or through one of the Test Automation Frameworks).

One of the main features supported by the MTF Python API is MQTT, which is an standard messaging protocol. It is designed as an extremely lightweight publish/subscribe messaging transport, ideal for connecting remote devices with a small code footprint and minimal network bandwidth.

In our architecture MTF is the publisher client offering the following functionalities for the subscribers: - Start and stop frame transmissions - Set and check signals over CAN, LIN and Flexray frames - Set and check Ethernet signals over SOME/IP and NPDU - Set and check hardware I/O signals - Set and check the voltage and current of the power supply - Start, suspend, resume and stop ECU simulations - Control simulations using predefined controllers

Robot

Robot Framework is an environment used to develop human-readable test cases. This is accomplished using keywords that transform the Python-API methods and functions to human-readable sentences.

This concept is particularly helpful for projects involving users with limited coding experience, as it is easy to learn.

However, there is a downside. New projects often require their specific keywords, which means extra work at the start, but it speeds up daily test case writing once they are set up.

We also provide a Test Definition Environment named TECHNICA-IDE- based on an IDE named RED Tool - for easier interaction with keywords. It provides documentation, list of keywords, drag-and-drop functionality as well as running capabilities.

xTR

If the project includes users with coding experience, they can write the test case directly in Python. This option means full API without requiring any keyword, and enables them to access more low level details.

To collect, organize and run test cases, we offer our test runner, xTR, built on the well-known unittest. xTR equips users with tools to execute test cases and configure various test benches as needed.

Note

To write your Python scripts, you can use your chosen IDE (Integrated Development Environment).

Please refer to the netAccFeatures-section section for test case examples.

MTF UI

MTF UI (MTF User Interface) is a testing and analysis tool which support various communication technologies such as CAN, LIN, FlexRay and Ethernet.

User can load different types of databases (arxml, xml, dbc, ldf, db, …) and configure channels manually or using a yaml file (same format supported by MTF). Once a configuration is generated, user can do :

  • offline and online analysis using a customized Wireshark that takes into consideration the databases and channels for each created configuration. It gives also the possibility to apply some additional rules on the trace to be analyzed.

  • online analysis with a tool named System Monitor that provides a fixed view : showing the last packet for each Id

MTF UI provides also other tools :

  • File Converter that handle different types of trace files.

  • Pcap Player / Pcap Recorder to switch between offline and online analysis.

  • Traffic Generator / Burst sending to send traffic and do stress tests.

MTF usage examples

Hardware in the Loop Testing

Technica HIL Solutions

The I/O lines of the System Under Test (SUT) are connected to the Modular Test System (MTS) which is connected to the PC where MTF Toolchain is running using Gigabit Ethernet connection.

Modular Test Framework or MTF Toolchain executes the automated tests and simulates the rest of the network nodes.

MTF UI can be connected to the MTF Core Module ( Inter-Process Communication) in order to visualize signals, control unit memory variables, etc.

Automation with MTF Toolchain

Using xTR

Test case language

Python

Category
Acceptance Testing
Recommended for Protocol Testing.

General info
xTR is Python’s xUnit style framework. It supports tests
automation and aggregation, test suites configuration and
execution and report generation.

Summary

  • Low-level

  • Code-driven approach

  • More control and flexibility over the test logic and execution
Using Robotframework

Test case language

Plain text syntax (https://robotframework.org/)

Category
Acceptance Testing
Recommended for Functional Testing

General info
Robot is an automated test framework
which has a simple plain text syntax and
can be extended easily with Python.

Summary

  • High-level

  • Keyword-driven approach

  • Abstract away the technical details and
    make test cases more readable and maintainable