FAIL

cockpit architecture check

1 error, 0 warnings. Trace demo/cockpit/traces/camera-dies.kev: QNX 8.0.0 on x86pc, 2 CPUs, 6.00 s, 44,054 events.

Architecture

display to camera: 19 messagesdisplay to logger: 19 messagesdisplay to nav: 20 messagesnav to gps: 20 messagesnav to logger: 20 messagesgpsnavcameraloggerdisplay
  • allowed, used
  • allowed, not seen
  • not in contract
  • deadlock loop
  • path in a finding

Findings

errorisolation

camera died and took display down with it

The contract says display must keep working when camera fails (at least 3 completed messages). camera died; display died 33 ms later.

Evidence: trace events 91117, 95162

Show the 38 trace events behind this
EventTimeWhat happened
666412621.162 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
666512621.219 mscamera[tid 1] MsgReceive got the message from display[tid 1]
666762622.957 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
671142679.347 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
671242679.407 mscamera[tid 1] MsgReceive got the message from display[tid 1]
671342679.489 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
674002738.653 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
674102738.715 mscamera[tid 1] MsgReceive got the message from display[tid 1]
674202738.950 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
679352794.192 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
679452794.236 mscamera[tid 1] MsgReceive got the message from display[tid 1]
679552794.325 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
682512849.696 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
682612849.762 mscamera[tid 1] MsgReceive got the message from display[tid 1]
682712849.870 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
696332904.072 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
696542904.358 mscamera[tid 1] MsgReceive got the message from display[tid 1]
696642904.464 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
700022959.562 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
700122959.620 mscamera[tid 1] MsgReceive got the message from display[tid 1]
700222959.697 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
704103014.937 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
704203014.986 mscamera[tid 1] MsgReceive got the message from display[tid 1]
704303015.075 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
694843070.275 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
694943070.339 mscamera[tid 1] MsgReceive got the message from display[tid 1]
695043070.430 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
729193124.610 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
729293124.667 mscamera[tid 1] MsgReceive got the message from display[tid 1]
729393124.756 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
753473187.457 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
753573187.521 mscamera[tid 1] MsgReceive got the message from display[tid 1]
753673187.614 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
831513240.667 msdisplay[tid 1] MsgSend to camera (coid 0x40000002, chid 1) message '\x03\x10'
831613240.688 mscamera[tid 1] MsgReceive got the message from display[tid 1]
831713240.732 mscamera[tid 1] MsgReply to display[tid 1] reply '\x03\x10'
911173274.794 msprocess camera (pid 1376282) destroyed
951623304.806 msprocess display (pid 1400857) destroyed
Why, according to claude-opus-5-5 confidence high

camera served every request normally (last reply at 3240.7 ms) and was then destroyed at 3274.8 ms. On display's next tick, MsgSend() on the dead camera connection returned -1. The camera-dies regression in run_display() treats that error as fatal ('camera failed ... giving up', return 1), so display exited 30 ms after camera instead of continuing to draw nav/logger frames.

At fault: MsgSend in display (error return from dead camera treated as fatal)

Fix: Do not exit when MsgSend to camera fails. Degrade to a 'no camera' frame and keep the nav/logger loop running. Better still, build with COCKPIT_FIXED=1 so the camera is read from its own camera_reader thread, which also protects display from a hung camera (a send timeout would not, because of _NTO_CHF_UNBLOCK on name_attach channels).

Proposed patch
--- a/cockpit.c
+++ b/cockpit.c
@@ run_display
-        } else if (ask(camera, MSG_FRAME, seq, "frame?", &frame) == -1) {
-            fprintf(stderr, "display: camera failed: %s, giving up\n", strerror(errno));
-            return 1;
-        }
+        } else if (camera == -1 || ask(camera, MSG_FRAME, seq, "frame?", &frame) == -1) {
+            if (camera != -1) {
+                fprintf(stderr, "display: camera failed: %s, continuing without it\n", strerror(errno));
+                ConnectDetach(camera);
+                camera = -1;
+            }
+            snprintf(frame.text, sizeof(frame.text), "no camera");
+        }

Cites events 83151, 83161, 83171, 91117, 95162 · Docs: MsgSend Synchronous message passing name_open name_attach

Message traffic

ClientServerIn contractMessagesRepliedErrorsUnansweredMedian replyWorst replyBudget
displaycamerayes1919000.15 ms1.80 ms20 ms
displayloggeryes1919000.10 ms0.27 ms20 ms
displaynavyes2020000.73 ms2.79 ms50 ms
navgpsyes2020000.17 ms0.87 ms20 ms
navloggeryes2020000.08 ms0.14 ms20 ms