pytest automated test - My point of view on test case timeout processing

1 pytest automated test - My point of view on test case timeout processing

1.1 Background

During the execution of the use case, the network may be waiting for some special files, and due to some exceptions, these conditions cannot be met and the use case is stuck. This situation is not allowed in automated testing and will waste a lot of waste. The time of the test task affects the progress of the test task and even leads to the extension of the release cycle of the version.
In order to set a timeout time for each use case, there are several methods:

1.2 Method 1: Add to each use case`@`Modifier

@It does not work well under Windows. If 1 use case timed out, the entire test task will be interrupted, which is very nonsense!

import pytest
 import time

 @(5)
 def test_case1():
     (2)
     assert True

 @(5)
 def test_case2():
     (6) # This use case will time out
     assert True

 def test_case3():
     print("User Case 3: Not executed under Windows")
     assert 3 == 3

1.3 Method 2: Use the hook function to add a timeout modifier to each use case

However,The following method is of no use in Windows!, the use case that timed out was also successfully executed!

import pytest
 import time

 def pytest_collection_modifyitems(session, config, items):
     for item in items:
         item.add_marker((3))

 def setup_function():
     print()

 def test_case3():
     print("Use Case 3: Start")
     (3)
     assert True
     print("Use Case 3: End")

 def test_case4():
     print("Use Case 4: Start")
     (5) # This use case will time out, but Windows does not, which is amazing!
     assert True
     print("Use Case 4: End")

 def test_case5():
     print("Use Case 5: Start")
     assert 3 == 3
     print("Use Case 5: End")

Execution results under Windows:

======================================================  ==============
 platform win32 -- Python 3.13.1, pytest-8.3.4, pluggy-1.5.0
 rootdir: D:\TYYSOFT\Study\Python\pytest
 configfile:
 plugins: check-2.4.1, html-4.1.1, metadata-3.1.1, timeout-2.3.1, xdist-3.6.1
 collected 3 items

 test_timeout_004.py
 Use Case 3: Start
 Use Case 3: End
 .
 Use Case 4: Start
 Use Case 4: End
 .
 Use Case 5: Start
 Use Case 5: End
 .

 -- Generated html report: file:///D:/TYYSOFT/Study/Python/pytest/ --
 ==================================== 3 passed in 8.03s ===================  ====================

Other so-called usetimeoutThe timeout process of markers is of no use under Windows!

1.4 Method 3: Introduce a method that uses threads to handle use cases timeouts

The timeout of the use case is actually mainly to monitor the timeout of each task, and by creating a basic functionbase_funcEach time this function is called, it creates a thread and sets a timeout time for the thread. The execution statements in the use case are passed through this thread functionbase_funcTo execute,base_funcThe timeout task will fail.

import pytest
 import threading
 import time

 # Core function: This function creates 1 thread and executes the given task. After the timeout, the use case will be set to fail.
 def base_func(task, timeout=5):
     thread = (target=task)
     ()
     (timeout) #Set timeout

     if thread.is_alive():
         print(f"::{task.__name__}: Statement execution timeout...")
         assert False

 # 1 long-term mission
 def long_time_task():
     (2)

 # The basic function of sending messages to the module
 def send_msg(msg):
     print(msg)

 # Preset conditions
 def setup_function():
     print()
     base_func(print("Preset condition!"))

 # This is use case 1
 def test_timeout_001():
     base_func(send_msg("use case 1 start"))
     base_func(long_time_task)
     base_func(send_msg("User Case 1 End"))

 def test_timeout_002():
     base_func(send_msg("use case 2 start"))
     base_func(send_msg("User Case 2 end"))

Output result:

======================================================  ==============
 platform win32 -- Python 3.13.1, pytest-8.3.4, pluggy-1.5.0
 rootdir: D:\TYYSOFT\Study\Python\pytest
 configfile:
 plugins: check-2.4.1, html-4.1.1, metadata-3.1.1, timeout-2.3.1, xdist-3.6.1
 collected 2 items

 test_timeout_002.py
 Preset conditions!
 Use Case 1 Begins
 ::long_time_task: Statement execution timeout...
 F
 Preset conditions!
 Use Case 2 Begins
 Use Case 2 ends
 .

 ====================================== FAILURES ======================  ============================
 ______________________________ test_timeout_001 _______________________________

     def test_timeout_001():
         base_func(send_msg("use case 1 start"))
 > base_func(long_time_task)

 test_timeout_002.py:31:
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

 task = <function long_time_task at 0x000001AAE4112DE0>, timeout = 5

     def base_func(task, timeout=5):
         thread = (target=task)
         ()
         (timeout) #Set timeout
    
         if thread.is_alive():
             print(f"::{task.__name__}: Statement execution timeout...")
 > assert False
 E assert False

 test_timeout_002.py:13: AssertionError
 -- Generated html report: file:///D:/TYYSOFT/Study/Python/pytest/ --
 =======================================================  =========
 FAILED test_timeout_002.py::test_timeout_001 - assert False
 =============================== 1 failed, 1 passed in 5.11s ======================  =========

Both use cases were executed, and the second use case did not abort the execution because the first use case timed out!

Use basic functions to wrap the need to test the execution statement, because when actually testing is executed, each statement may execute timeout, encapsulating execution can ensure the rigor of the test script.

Author statement: This article is used to record and share the author's learning experience. Some texts or examples may come from AI platforms, such as: Doubao,DeepSeek (Silicon-based flow)(Register link)etc. Due to my limited level, there are inevitably mistakes in expression. Welcome to leave a message and communicate and give advice!
Copyright © 2022~2025 All rights reserved.