oneM2MTester

BSD 3-Clause License

Copyright (c) 2017, IoTKETI All rights reserved Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Welcome to oneM2M Conformance Testing Project Wiki Page!!

Introduction

oneM2MTester project is a oneM2M conformance testing project which is initialized by Korea Electronics Technology Institute (KETI) in January 2016 and collaborated with 10 international members Sejong University (SJU), Ericsson, Easy Global Market (EGM), Huawei, Korea Telecommunications Technology Association (TTA), DTNC, Sensinov, InterDigital, LAAS-CNRS, and InnoWireless.

The project aims to develop and distribute an open source oneM2M conformance testing tool and associated user manuals for oneM2M implementation (CSE and AE) developers and testing engineers to test and evaluate the compliance of function implementation with oneM2M specifications.

Functionality

The oneM2MTester implements system adapters and codecs within a port to enable the communication between TTCN-3 test system and the implementation under test (IUT). The system adapter and codec are protocol dependent, and responsible for encoding oneM2M request primitive message into protocol-specific formatted message before sending the messages to the IUT and vice versa.

The oneM2MTester makes use of Eclipse TITAN’s TTCN-3 Test System environment which includes components of TITAN Designer, TITAN Executor, and TITAN LogViewer etc.

•     TITAN Designer    : providing TTCN-3 editing and grammar checking functions as well as a graphical user interface for building an executable test suite;
•     TITAN Executor    : providing a graphical user interface to execute testcases and generating corresponding verdict result;
•     TITAN LogViewer : logging the testcase running progress and providing a graphical presentation to assist user debugging their IUT. Besides, a group of log options are available for users to choose for different log purposes;
•     TITAN Runtime Configuration graphical user interface : providing protocol configuration for IUT and logging preferences for testers.

Software Architecture

The oneM2MTester extends the architecture of Eclipse TITAN by adding oneM2M system adapters and codecs for oneM2M binding protocols to enable that the TTCN-3 test system can communicate with and execute oneM2M Abstract Test Suite against oneM2M implementations to achieve testing and evaluation purpose.

The oneM2MTester implements communication ports that supports “translation capabilities” feature which is defined as an extension feature of TTCN-3 language as following:

TTCN-3 language extensions: Configuration and Deployment Support : “allows to convert messages and/or addresses of one type into messages and/or addresses of different type during sending or receiving… It can be used e.g. in situations where the test behaviour is defined on one set of data types but the implementation under test (or connected component) actually communicates using a different set of data types, i.e. if the test system works on a different layer of the protocol stack than the implementation of the test.”

oneM2M Abstract Test Suite

• oneM2M Abstract Test Suite (hereafter ATS) is a standardized Conformance testing test suite that are contributed and maintained by oneM2M Task Force-001that is supervised by oneM2M Test Work Group.
• oneM2M ATS consists of a suite of testcases that are written in TTCN-3 standardized testing language, and the testcases are developed following test objectives and test behaviours that are defined in oneM2M specification TS-0018 – Test Suite Structure and Test Purpose.
• oneM2M ATS source code is available on oneM2M ATS GitLab project.

Runtime Environment

Note: WE STRONGLY RECOMMEND USERS TO INSTALL RUNTIME ENVIRONMENT IN LINUX OPERATION SYSTEM. THE GUIDE FOR SETTING RUNTIME ENVIRONMENT ARE APPLIED FOR LINUX OPERATION SYSTEM FOR THE MOMENT!!**

To run oneM2MTester in your Linux/Windows system, users have to install tools, and plugins as following: 

• Eclipse Integrated Development Environment (IDE)

         Eclipse IDE is required to install Eclipse TITAN environment. We recommend users to check and download the latest Eclipse IDE unless there is explicitly declaration statement for the compatiability issue among oneM2MTester and specific Eclipse IDE .

• Eclipse TITAN Core

TITAN Core is a core part of Eclipse TITAN, which is responsible for executing in a Unix/Linux-like environment and a set of Eclipse plug-ins. TITAN Core implements TTCN-3 Compiler, MakeFile generator etc.

The TITAN compiler builds an executable test suite from the TTCN-3 modules, and the TITAN runtime library. It is written in C++ and is available for downloading in binary form or git clone from Eclipse Titan GitHub. Installation of TITAN core for Linux/Window can be found from README.

After installation of TITAN core, users need to configure several environment variables as following:

   – TTCN3_DIR: This variable contains the absolute location of the TITAN installation.

          – LD_LIBRARY_PATH: Set to $TTCN3_DIR/lib to configute the path of OpenSSL shared library that is provided with TITAN.

          – PATH: Set to $TTCN3_DIR/bin to configure the path of TITAN binaries.    

More detail regarding the configuration for system variables, please refer to Installation Guide for the TITAN TTCN-3 Test Executor.

• Eclipse TITAN Plugins

Eclipse TITAN provides a suite of plugins to facilitate the TTCN-3 coding, test execution control and management, logging of test execution as well as generation of test verdict.

To simplify the installation of TITAN plugins into Eclipse IDE, the plugins as well as required dependencies are compressed into one single Titan plugin installation package. Users can download it from Eclipse TITAN project download page.

After installation of TITAN plugins into Eclipse IDE, the installation path of the TITAN Core shall be configured in TITAN Preference of the TITAN Editor view.

Get Started

Get start your experience with oneM2MTester following steps:

• Clone/Download Source Package

Clone oneM2MTester source package by running command  https://github.com/IoTKETI/oneM2MTester.git  Or you can directly download the package from the oneM2MTester GitHub page.

• Import oneM2MTester source package into Workspace
  • Create a new TITAN Project and import all files within the downloaded oneM2MTester source package, or
  • Import directly the downloaded oneM2MTester source package

• Build oneM2MTester project via Command Line (CLI) following below steps:
  1. Export oneM2MTester TITAN project into a TITAN Description file (.tpd) through TTCN-3 Editing view of Eclipse IDE
  2. Generate Makefile with the generated TITAN Description file (.tpd) e.g. oneM2MTester_v_2_0_0.tpd by running command  ttcn3_makefilegen -t oneM2MTester_v_2_0_0.tpd 
     • The generated Makefile is located in a binary folder /bin
  3. Go to the /bin directory and run command make to compile the TTCN-3 modules in the oneM2MTester project
     • Tip: User can specify the number of CPU cores to allocate them to compile your TITAN project in order to speed up the compilation progress make -j<NUMBER_OF_CPU_CORES>. E.G. make -j4 indicates that 4 cores is allocated to  perform the compilation pf TTCN-3 modules.

• Set IUT Configurations and TITAN Runtime Configurations
  • Set IUT configurations in a TITAN Configuration file (.cfg, e.g. TS_HTTP_config.cfg)
    • This configuration file overwrites parameters defined in oneM2M PIXIT modules, such as CSE-ID, IUT IP and Port etc.
    • Users might include a suite of testcases within the TITAN Configuration file. In this case, when users starts running button, those testcases will be automatically loaded from TITAN Configuration file to run.
  • Set TITAN runtime configurations in terms of TIATN parallel launcher configuration through Titan Runtime Configuration GUI including
    • Basic main controller options: set TITAN project information such as path of Executable, TITAN configuration etc.
    • Host controller : set by clicking init button
    • Test set : Users might generate a suite of test set containing a group of testcases in order to execute testcases in a group manner
    • Basic performance setting (default is recommended)
    • Environent setting
• Run Executable Test Suite
  • In TITAN Executor graphical user interface, testcases can be started by TITAN Execution Controller.
  • The TITAN Executor controls the test execution and generates final verdict for each executed testcase while TITAN LogViewer generates logs during the testcase execution.
• Check Verdict
  • Test verdict is generated at end of test suite execution to indicate an execution status of each testcase.
  •  In the case of “fail” or “inconc” verdict, the testers can refer to the running logs to figure out what causes the excution failure.

• Debug
  • TITAN provides a suite of debugging options to handle different debugging requirements.
  • The debug preferences can be configured in TITAN Configuration file.
  • The debug options include  
    • DEBUG
    • PORTEVENT
    • TESTCASE
    • STATISTICS
    • MATCHING
    • PARALLEL
    • VERDICTOP
    • VERDICTOP_SETVERDICT

More details for the debug options can be found in Eclipse TITAN Programmer’s Technical Reference Guide for the TITAN TTCN-3 Toolset.

Introduction

oneM2MTester is a oneM2M Conformance testing project with aims to develop and distribute an open source oneM2M Conformance testing tool as well as associated user manuals. The oneM2MTester is initialized by Korea Electronics Technology Institute (KETI) in January 2016, and it is a global collaboration project with 11 members as following:     

•   Korea Electronics Technology Institute (KETI), 
•   Sejong University (SJU), 
•   Ericsson,
•   Korea Telecommunications Technology Association (TTA),
•   Easy Global Market (EGM), 
•   Sensinov,   
•   InterDigital,
•   LAAS-CNRS,  
•   InnoWireless,   
•   DTNC, and 
•   Huawei.

The oneM2MTester Conformance testing tool is developed based on open source Eclipse TITAN, and is mainly designed for oneM2M platform developers and/or testing engineers to test and evaluate their oneM2M implementations (CSE and AE) during or after the implementation development. The oneM2MTester is capable of running oneM2M standardized abstract testcases against oneM2M implementations, and providing users testcase execution results and logs.