# Test System

* [Compare](/linux/ltp.md#compare)
  * [Docs](/linux/ltp.md#docs)
  * [Videos](/linux/ltp.md#videos)
* [Test Suites](/linux/ltp.md#test-suites)
  * [Linux Test Project](/linux/ltp.md#linux-test-project)
  * [LTP-DDT](/linux/ltp.md#ltp-ddt)
  * [Android](/linux/ltp.md#android)
  * [dev-tools](/linux/ltp.md#dev-tools)
    * [Selftests](/linux/ltp.md#selftests)
    * [KUnit - Kernel Unit Testing Framework](/linux/ltp.md#kunit---kernel-unit-testing-framework)
  * [Storage](/linux/ltp.md#storage)
    * [xfstests](/linux/ltp.md#xfstests)
    * [blktests](/linux/ltp.md#blktests)
* [Test Automation Tools](/linux/ltp.md#test-automation-tools)
  * [0Day](/linux/ltp.md#0day)
  * [KernelCI - Python](/linux/ltp.md#kernelci---python)
  * [LAVA - Python](/linux/ltp.md#lava---python)
  * [Fuego - Mitsubishi](/linux/ltp.md#fuego---mitsubishi)
  * [U-Boot “pytest suite”](/linux/ltp.md#u-boot-pytest-suite)
  * [tbot - Python](/linux/ltp.md#tbot---python)
  * [labgrid - Python](/linux/ltp.md#labgrid---python)
  * [r4d - Python](/linux/ltp.md#r4d---python)
  * [Yocto project / ptest](/linux/ltp.md#yocto-project--ptest)
  * [Kerneltests.org](/linux/ltp.md#kerneltestsorg)
  * [Robot Framework - Python](/linux/ltp.md#robot-framework---python)
  * [More](/linux/ltp.md#more)
* [Static Code Analyzers](/linux/ltp.md#static-code-analyzers)
  * [Sparse](/linux/ltp.md#sparse)
  * [clang / LLVM](/linux/ltp.md#clang--llvm)
  * [Coccinelle](/linux/ltp.md#coccinelle)
  * [smatch](/linux/ltp.md#smatch)
  * [Coverity](/linux/ltp.md#coverity)
* [Fuzzing Tools](/linux/ltp.md#fuzzing-tools)
  * [Trinity](/linux/ltp.md#trinity)
  * [Syzcaller](/linux/ltp.md#syzcaller)
  * [kcov - code coverage for fuzzing](/linux/ltp.md#kcov---code-coverage-for-fuzzing)

## Compare

### Docs

<https://elinux.org/Testing>\
<https://elinux.org/Test_Systems#Test_Projects>\
<https://elinux.org/images/9/9f/Linux-Kernel-Testing-Where-are-we.pdf> (2016) <http://events17.linuxfoundation.org/sites/events/files/slides/PRE-trunk-ELCE-Automation-beyond-Testing_1.pdf> (2017)

### Videos

<https://www.youtube.com/watch?v=NRywFwe0uwU>\
introduce multiple frameworks available to test your Embedded Linux System and compare the use-cases (2018)\
Slide: <https://elinux.org/images/0/08/Primer-Testing-Your-Embedded-System-What-is-a-ptest-Lava-Fuego-KernelCI-and...-Jan-Simon-Moeller-The-Linux-Foundation.pdf>

## Test Suites

### Linux Test Project

<https://github.com/linux-test-project/ltp/wiki>

<http://ltp.sourceforge.net/documentation/how-to/ltp.php#_3.2>

```
runltp
    filesystem stress tests
    disk I/O tests
    memory management stress tests
    ipc stress
    scheduler tests
    commands functional varification tests
    system call functional varification tests
networktests.sh
    networking related tests
diskio.sh
    tests related to floppy and CD-ROM drives
```

### LTP-DDT

<https://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/AM335X/linux/Foundational_Components_Kernel_LTP-DDT_Validation.html>\
LTP-DDT focuses on embedded device driver tests.

### Android

<https://android.googlesource.com/platform/external/ltp/>

### dev-tools

<https://www.kernel.org/doc/html/latest/dev-tools/>

#### Selftests

<https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html>

```
cd /linux/tools/testing/
fault-injection  
ktest  
kunit  
nvdimm  
radix-tree  
scatterlist  
selftests  
vsock

make -C selftests
make -C selftests run_tests
```

"The magical fantasy land of Linux kernel testing" - Russell Currey (LCA 2020): <https://www.youtube.com/watch?v=9Fzd6MapG3Y>

#### KUnit - Kernel Unit Testing Framework

<https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>

### Storage

#### xfstests

<https://kernel.googlesource.com/pub/scm/fs/ext2/xfstests-bld/+/HEAD/Documentation/what-is-xfstests.md>

```
xfs, ext2, ext4, cifs, btrfs, f2fs, reiserfs, gfs, jfs, udf, nfs, tmpfs
```

#### blktests

<https://github.com/osandov/blktests>\
<https://zonedstorage.io/tests/blktests/>

| Group name | Description                       |
| ---------- | --------------------------------- |
| block      | Block layer generic tests         |
| loop       | Loopback device tests             |
| meta       | blktests self tests               |
| nbd        | Network block device driver tests |
| nvme       | NVMe driver tests                 |
| nvmeof-mp  | NVME-over-fabrics multipath tests |
| scsi       | SCSI layer tests                  |
| srp        | SCSI RDMA Protocol driver tests   |
| zbd        | Zoned block device tests          |

## Test Automation Tools

### 0Day

test suites used by 0-Day performance test and LKP test tool: <https://01.org/lkp/documentation/0-day/lkp-test-suites-description>

### KernelCI - Python

<https://kernelci.org/> | Linux Foundation project\
use LAVA with KernelCI: <https://github.com/kernelci/kernelci-core/blob/main/doc/lava.md>

### LAVA - Python

Linaro Automated Validation Architecture | <https://www.lavasoftware.org/>\
Source: <https://git.lavasoftware.org/lava/lava>\
Docker: <https://github.com/kernelci/lava-docker/>

### Fuego - Mitsubishi

<https://elinux.org/images/6/6c/Introduction-to-Fuego-JJ58-1.pdf>\
Fuego = (Jenkins + abstraction scripts + pre-packaged tests) inside a container

<https://elinux.org/images/d/d4/ELCE-2016-Continuous-Integration-and-Autotest-Environment-using-Fuego_Master.pdf>

Source: <https://bitbucket.org/fuegotest/>

Introduction Videos:

* 53min <https://www.youtube.com/watch?v=AueBSRN4wLk>  &#x20;
* 28min <https://www.youtube.com/watch?v=2bSqzhTxLdU>

### U-Boot “pytest suite”

<https://github.com/u-boot/u-boot/tree/master/test/py>\
only for U-Boot (with build support)

### tbot - Python

<https://tbot.tools/>\
<https://github.com/Rahix/tbot>\
allow running tests in an automated setting (CI).

### labgrid - Python

<https://github.com/labgrid-project/labgrid#purpose>\
create an abstraction of the hardware control layer needed for testing of embedded systems, automatic software installation and automation during development

* pytest plugin to write tests for embedded systems connecting serial console or SSH
* remote client-exporter-coordinator infrastructure to make boards available from different computers on a network
* power/reset management via drivers for power switches or onewire PIOs
* upload of binaries via USB: imxusbloader/mxsusbloader (bootloader) or fastboot (kernel)

Labgrid itself is not a testing framework, but is intended to be combined with pytest (and additional pytest plugins).

### r4d - Python

<https://github.com/ci-rt/r4d>\
r4d means 'Remote For Device-under-test' and is an infrastructure for power-control and console access for multiple Linux Boards that should be controlled by a test-infrastructure like jenkins.

### Yocto project / ptest

<https://wiki.yoctoproject.org/wiki/Ptest>\
Ptest (package test) is a concept for building, installing and running the test suites that are included in many upstream packages, and producing a consistent output format for the results.\
Source: <http://git.yoctoproject.org/clean/cgit.cgi/poky/tree/>

### Kerneltests.org

Created to test stable release candidates\
<https://kerneltests.org/builders>

### Robot Framework - Python

<http://robotframework.org/robotframework/#user-guide>

OperatingSystem | Process | Dialogs / String | Telnet

### More

<http://fuegotest.org/wiki/Other_test_systems>

<https://elinux.org/Test_Stack_Survey#Responses>

```
kisskb    
Olof's autobuilder: mainline and next for arm, arm64, powerpc
Olof's autobooter:  mainline, next, arm-soc
Tegra builds: mainline
Buildbot by Mark Brown: x86_64, arm, arm64 (8 builds)
CKI - used by RedHat
Gentoo Kernel CI
hottest notes
Krzk Samsung-SoC
ktest    
Labgrid
LKFT
Opentest
Phoronix
SLAV
syzbot (& syzkaller)
TCF
Xilinx test (aka regression_xlnx)
```

## Static Code Analyzers

### Sparse

<https://www.kernel.org/doc/html/latest/dev-tools/sparse.html>

<https://sparse.docs.kernel.org/en/latest/>\
the semantic parser, provides a compiler frontend capable of parsing most of ANSI C as well as many GCC extensions, and a collection of sample compiler backends, including a static analyzer also called sparse.

### clang / LLVM

<https://clang-analyzer.llvm.org/>\
the analyzer is part of Clang

### Coccinelle

<https://coccinelle.gitlabpages.inria.fr/website/>

### smatch

<https://github.com/error27/smatch>\
a semantic parser of source files

### Coverity

Commercial Static Analyzer： <https://scan.coverity.com/o/oss_success_stories>

## Fuzzing Tools

Paper: <https://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf>

### Trinity

<https://github.com/kernelslacker/trinity>\
a system call fuzzer which employs some techniques to pass semi-intelligent arguments to the syscalls being called.

### Syzcaller

<https://github.com/google/syzkaller/blob/master/docs/internals.md>

* Net: <https://github.com/google/syzkaller/blob/master/docs/linux/external_fuzzing_network.md>
* USB: <https://github.com/google/syzkaller/blob/master/docs/linux/external_fuzzing_usb.md>

### kcov - code coverage for fuzzing

<https://www.kernel.org/doc/html/latest/dev-tools/kcov.html>\
kcov exposes kernel code coverage information in a form suitable for coverage- guided fuzzing (randomized testing)

<https://www.netbsd.org/~kamil/Maciej_Grochowski-FS_Fuzzing_EuroBSDCon2019.pdf>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://book.ferro.pro/linux/ltp.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.