isolationcamera stopped answering before the trace started and took display down with it
The contract says display must keep working when camera fails (at least 3 completed messages). camera stopped answering before the trace started; display completed only 0 messages afterwards and its thread 1 is still REPLY-blocked on camera.
Evidence: trace events none
camera's handler wedged (camera-hang fault: after 5 frames camera_handle loops in pause() and never calls MsgReply), and this happened before the trace started. display calls MsgSend to camera synchronously from its only drawing thread, so thread 1 stays REPLY-blocked on camera forever and never reaches nav or logger again (0 completed messages, contract needs 3). A send timeout would not help: the name_attach channel has _NTO_CHF_UNBLOCK, so a timed-out client stays blocked until the wedged camera handles the unblock pulse, which it never does.
At fault: MsgSend in display (blocking call to camera from the main draw loop); MsgReply missing in camera
Fix: Move the camera MsgSend out of the display loop into a dedicated reader thread. The loop draws the last frame that thread delivered and shows "camera stale" when the frame is old, so a hung or dead camera only blocks the reader thread. The reader thread also exits cleanly on error, so the display does not exit with a dead camera. This is the existing COCKPIT_FIXED path: make it the unconditional behaviour, or run with COCKPIT_FIXED=1. Separately, fix camera's frame grabber so it always replies, or use MsgError on failure.
Proposed patch
--- a/cockpit.c
+++ b/cockpit.c
@@ run_display
- if (fixed) {
- pthread_t reader;
- pthread_create(&reader, NULL, camera_reader, &camera);
- }
+ /* camera is allowed to fail: never MsgSend to it from the draw loop */
+ pthread_t reader;
+ pthread_create(&reader, NULL, camera_reader, &camera);
@@
- if (fixed) {
- pthread_mutex_lock(&frame_lock);
- if (++frame_age > 3) {
- snprintf(frame.text, sizeof(frame.text), "camera stale");
- } else {
- snprintf(frame.text, sizeof(frame.text), "%s", last_frame);
- }
- pthread_mutex_unlock(&frame_lock);
- } else if (ask(camera, MSG_FRAME, seq, "frame?", &frame) == -1) {
- fprintf(stderr, "display: camera failed: %s, giving up\n", strerror(errno));
- return 1;
- }
+ pthread_mutex_lock(&frame_lock);
+ if (++frame_age > 3) {
+ snprintf(frame.text, sizeof(frame.text), "camera stale");
+ } else {
+ snprintf(frame.text, sizeof(frame.text), "%s", last_frame);
+ }
+ pthread_mutex_unlock(&frame_lock);
Cites events · Docs: MsgSend MsgReply Synchronous message passing name_attach TimerTimeout
Unverified: the explanation cited no trace events.