FAIL

cockpit architecture check

1 error, 1 warning. Trace demo/cockpit/traces/camera-hang-late.kev: QNX 8.0.0 on x86pc, 2 CPUs, 3.00 s, 7,972 events.

Architecture

display to camera: 0 messagesdisplay to logger: 0 messagesdisplay to nav: 0 messagesnav to gps: 0 messagesnav to logger: 0 messagesgpsnavcameraloggerdisplay
  • allowed, used
  • allowed, not seen
  • not in contract
  • deadlock loop
  • path in a finding

Findings

  1. camera stopped answering before the trace started and took display down with it isolationstart here
  2. camera took a message from display and never replied stuck-reply
errorisolation

camera stopped answering before the trace started and took display down with it

The contract says display must keep working when camera fails (at least 3 completed messages). camera stopped answering before the trace started; display completed only 0 messages afterwards and its thread 1 is still REPLY-blocked on camera.

Evidence: trace events none

Why, according to claude-opus-5-5 confidence medium

camera's handler wedged (camera-hang fault: after 5 frames camera_handle loops in pause() and never calls MsgReply), and this happened before the trace started. display calls MsgSend to camera synchronously from its only drawing thread, so thread 1 stays REPLY-blocked on camera forever and never reaches nav or logger again (0 completed messages, contract needs 3). A send timeout would not help: the name_attach channel has _NTO_CHF_UNBLOCK, so a timed-out client stays blocked until the wedged camera handles the unblock pulse, which it never does.

At fault: MsgSend in display (blocking call to camera from the main draw loop); MsgReply missing in camera

Fix: Move the camera MsgSend out of the display loop into a dedicated reader thread. The loop draws the last frame that thread delivered and shows "camera stale" when the frame is old, so a hung or dead camera only blocks the reader thread. The reader thread also exits cleanly on error, so the display does not exit with a dead camera. This is the existing COCKPIT_FIXED path: make it the unconditional behaviour, or run with COCKPIT_FIXED=1. Separately, fix camera's frame grabber so it always replies, or use MsgError on failure.

Proposed patch
--- a/cockpit.c
+++ b/cockpit.c
@@ run_display
-    if (fixed) {
-        pthread_t reader;
-        pthread_create(&reader, NULL, camera_reader, &camera);
-    }
+    /* camera is allowed to fail: never MsgSend to it from the draw loop */
+    pthread_t reader;
+    pthread_create(&reader, NULL, camera_reader, &camera);
@@
-        if (fixed) {
-            pthread_mutex_lock(&frame_lock);
-            if (++frame_age > 3) {
-                snprintf(frame.text, sizeof(frame.text), "camera stale");
-            } else {
-                snprintf(frame.text, sizeof(frame.text), "%s", last_frame);
-            }
-            pthread_mutex_unlock(&frame_lock);
-        } else if (ask(camera, MSG_FRAME, seq, "frame?", &frame) == -1) {
-            fprintf(stderr, "display: camera failed: %s, giving up\n", strerror(errno));
-            return 1;
-        }
+        pthread_mutex_lock(&frame_lock);
+        if (++frame_age > 3) {
+            snprintf(frame.text, sizeof(frame.text), "camera stale");
+        } else {
+            snprintf(frame.text, sizeof(frame.text), "%s", last_frame);
+        }
+        pthread_mutex_unlock(&frame_lock);

Cites events · Docs: MsgSend MsgReply Synchronous message passing name_attach TimerTimeout

Unverified: the explanation cited no trace events.

warningstuck-reply

camera took a message from display and never replied

display thread 1 has been REPLY-blocked on camera for the whole trace (3005 ms) and was already blocked when it started (budget 20.0 ms). The server is not blocked on another server; the trace does not show what it is doing. The contract plans for this server failing, see the isolation check.

Evidence: trace events none

Show the 1 trace events behind this
EventTimeWhat happened
-10000000 (note)-pidin: display pid 938008 tid 1 REPLY-blocked on camera pid 913433 (state when the trace stopped)

Message traffic

ClientServerIn contractMessagesRepliedErrorsUnansweredMedian replyWorst replyBudget