FAIL

cockpit architecture check

1 error, 0 warnings. Trace demo/cockpit/traces/slow-reply.kev: QNX 8.0.0 on x86pc, 2 CPUs, 6.00 s, 35,517 events.

Architecture

display to camera: 14 messagesdisplay to logger: 14 messagesdisplay to nav: 15 messagesnav to gps: 15 messagesnav to logger: 14 messagesgpsnavcameraloggerdisplay
  • allowed, used
  • allowed, not seen
  • not in contract
  • deadlock loop
  • path in a finding

Findings

errorreply-budget

nav answers display too slowly

14 of 14 replies from nav to display took longer than 50 ms; worst 222.3 ms. Times come from the trace clock; under emulation they are inflated, so budgets should be generous.

Where the slowest reply (222.3 ms) went
  • 204.9 ms server sleeping (NANOSLEEP)
  • 9.0 ms server running
  • 4.4 ms server waiting on logger
  • 3.6 ms server preempted · io-sock at priority 21 1.5 ms, logger at priority 10 1.5 ms, procnto-smp-instr at priority 254 0.4 ms
  • 0.2 ms queued, server busy elsewhere
  • 0.2 ms server waiting on gps · there: queued, server busy elsewhere 0.1 ms

Evidence: trace events 64268, 70867, 56202

Show the 45 trace events behind this
EventTimeWhat happened
562022216.631 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
562122216.662 msnav[tid 1] MsgReceive got the message from display[tid 1]
564412427.024 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
577452489.414 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
577552489.570 msnav[tid 1] MsgReceive got the message from display[tid 1]
616762692.877 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
618602748.070 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
618702748.364 msnav[tid 1] MsgReceive got the message from display[tid 1]
621122951.696 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
623243005.559 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
623343005.708 msnav[tid 1] MsgReceive got the message from display[tid 1]
624983212.505 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
634003268.385 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
634103268.537 msnav[tid 1] MsgReceive got the message from display[tid 1]
640973472.329 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
642683528.287 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
642783528.528 msnav[tid 1] MsgReceive got the message from display[tid 1]
680743750.628 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
694313805.171 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
694413805.437 msnav[tid 1] MsgReceive got the message from display[tid 1]
696444011.757 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
708674069.427 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
708774070.092 msnav[tid 1] MsgReceive got the message from display[tid 1]
710674280.587 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
714104338.465 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
714204338.661 msnav[tid 1] MsgReceive got the message from display[tid 1]
716274541.237 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
759584597.397 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
759684597.552 msnav[tid 1] MsgReceive got the message from display[tid 1]
763874799.321 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
765754858.809 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
765854858.950 msnav[tid 1] MsgReceive got the message from display[tid 1]
778355063.109 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
780865121.419 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
780965121.570 msnav[tid 1] MsgReceive got the message from display[tid 1]
782605323.408 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
784995375.990 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
785095376.158 msnav[tid 1] MsgReceive got the message from display[tid 1]
786865578.767 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
793935632.179 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
794035632.321 msnav[tid 1] MsgReceive got the message from display[tid 1]
835255838.224 msnav[tid 1] MsgReply to display[tid 1] reply '\x02\x10'
837605891.507 msdisplay[tid 1] MsgSend to nav (coid 0x40000001, chid 1) message '\x02\x10'
837705891.687 msnav[tid 1] MsgReceive got the message from display[tid 1]
-83761 (note)-at trace end display[tid 1] is still REPLY-blocked on nav (no reply to event 83760)
Why, according to claude-opus-5-5 confidence high

nav_handle() sleeps 200 ms (the COCKPIT_FAULT=slow-reply branch, sleep_ms(200)) between MsgReceive() and MsgReply(). Every round trip therefore takes about 203-222 ms against a 50 ms budget. Because MsgSend() is synchronous, display stays REPLY-blocked for the whole time and its frame loop stalls with it. Priority is not a factor: nav is sleeping, not being preempted.

At fault: MsgReply delayed in nav (nav_handle sleeps 200 ms before replying to display's MsgSend)

Fix: Remove the 200 ms sleep from nav's request path so nav replies as soon as it has the gps position. If nav really needs 200 ms of work per route, move that work out of the handler: compute routes in a background thread and reply with the latest one. Only raise reply_budget_ms.nav if the contract is wrong, and a 200 ms route lookup is not acceptable for a display ticking every 50 ms.

Proposed patch
--- a/cockpit.c
+++ b/cockpit.c
@@ static int nav_handle(int rcvid, cockpit_msg_t *msg, cockpit_msg_t *reply) {
     cockpit_msg_t pos;
     gps_position(nav_gps, msg->seq, "where?", &pos);
-    if (has_fault("slow-reply")) {
-        sleep_ms(200);
-    }
     snprintf(reply->text, sizeof(reply->text), "route via %.36s", pos.text);
     log_line(nav_logger, msg->seq, "route sent");
     return 0;
 }

Cites events 56202, 56212, 56441, 57745, 57755, 61676, 64268, 64278, 68074, 83760, 83770, -83761 · Docs: MsgSend MsgReceive MsgReply Synchronous message passing

Message traffic

ClientServerIn contractMessagesRepliedErrorsUnansweredMedian replyWorst replyBudget
displaycamerayes1414000.19 ms1.53 ms20 ms
displayloggeryes1414000.10 ms1.25 ms20 ms
displaynavyes151401204.12 ms222.34 ms50 ms
navgpsyes1515000.20 ms1.45 ms20 ms
navloggeryes1414000.24 ms0.73 ms20 ms