HPMC User Guide v 1.00
© 2022 Bassem W. Jamaleddine
Your HPMC comes with the dint program that display the system interrupts. dint is a command that you can issue at the command prompt. Since your HPMC has hundreds of cores, and your Xeon Phi MIC has as well hundreds of cores, you need a utility that can display the system interrupts while focusing on the busiest cores.
The chapter "Programming and Running by MIC Native Loading" showed how to run diffusion_omp by offloading it to the MIC. In the following we show how to use the dint command to display the interrupts as the program is executing.
The interrupt names may be different for each system. To display the names of the interrupts, you can simply use dint with the -info option:
# dint -info
dint can display all cores. Use -transpose option so that the 228 cores are displayed one per row, a neater view.
# (HPMC9) 03:59 root@mic0: /tools # ./dint -transpose -iter 7 -delay 1 -interrupts LOC,RES,CAL,4,5,26,TLB
Use the -prune option to display the cores by pruning a range between the beginning and the end:
# (HPMC9) 03:59 root@mic0: /tools # ./dint -transpose -iter 7 -delay 1 -interrupts LOC,RES,CAL,4,5,26,TLB -prune 3,5
Displaying all the cores can be distracting, so to focus only on a range of cores, just add in -sortby option:
# (HPMC9) 04:02 root@mic0: /tools # ./dint -transpose -iter 7 -delay 1 -interrupts LOC,RES,CAL,4,5,26,TLB -prune 3,5 -sortby RES
To print all cores focusing on the RES interrupt, use this command:
# (HPMC9) 03:59 root@mic0: /tools # ./dint -transpose -iter 7 -delay 1 -interrupts LOC,RES,CAL,4,5,26,TLB -sortby RES
The following shows the dint command running on the MIC of hpmc9, where we focused on five cores that are busy with the RES (Rescheduling interrupts). After each iteration the busiest cores with the RES are displayed first.
# (HPMC9) 03:40 root@mic0: /tools # ./dint -NVU 5 -iter 7 -delay 1 -interrupts LOC,RES,CAL,4,5,26,TLB -sortby RES
(HPMC9) 03:40 root@mic0: /tools # ./dint -NVU 5 -iter 7 -delay 1 -interrupts LOC,RES,CAL,4,5,26,TLB -sortby RES
CPU227 CPU0 CPU225 CPU167 CPU26 summed
2026-04-23 03:41:04 LOC 70 121 80 162 162 595 Local timer interrupts
2026-04-23 03:41:04 RES 12 8 3 2 2 27 Rescheduling interrupts
2026-04-23 03:41:04 CAL 84 0 47 0 0 131 Function call interrupts
2026-04-23 03:41:04 4 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:04 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:04 26 0 0 0 0 0 0 SBOX-ICR-edge vnet intr
2026-04-23 03:41:04 TLB 0 0 1 0 0 1 TLB shootdowns
CPU226 CPU227 CPU0 CPU225 CPU119 summed
2026-04-23 03:41:06 LOC 118 132 150 62 159 621 Local timer interrupts
2026-04-23 03:41:06 RES 12 9 8 7 3 39 Rescheduling interrupts
2026-04-23 03:41:06 CAL 0 4 8 93 0 105 Function call interrupts
2026-04-23 03:41:06 4 0 0 2 0 0 2 IO-APIC-edge dma channel
2026-04-23 03:41:06 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:06 26 0 0 4 0 0 4 SBOX-ICR-edge vnet intr
2026-04-23 03:41:06 TLB 0 0 0 1 0 1 TLB shootdowns
CPU227 CPU0 CPU225 CPU226 CPU10 summed
2026-04-23 03:41:07 LOC 96 101 40 71 156 464 Local timer interrupts
2026-04-23 03:41:07 RES 10 9 5 4 2 30 Rescheduling interrupts
2026-04-23 03:41:07 CAL 33 53 47 0 0 133 Function call interrupts
2026-04-23 03:41:07 4 0 2 0 0 0 2 IO-APIC-edge dma channel
2026-04-23 03:41:07 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:07 26 0 3 0 0 0 3 SBOX-ICR-edge vnet intr
2026-04-23 03:41:07 TLB 0 0 1 2 0 3 TLB shootdowns
CPU0 CPU225 CPU227 CPU226 CPU134 summed
2026-04-23 03:41:09 LOC 99 149 98 101 160 607 Local timer interrupts
2026-04-23 03:41:09 RES 18 13 11 5 3 50 Rescheduling interrupts
2026-04-23 03:41:09 CAL 49 11 0 49 0 109 Function call interrupts
2026-04-23 03:41:09 4 2 0 0 0 0 2 IO-APIC-edge dma channel
2026-04-23 03:41:09 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:09 26 4 0 0 0 0 4 SBOX-ICR-edge vnet intr
2026-04-23 03:41:09 TLB 1 0 1 1 0 3 TLB shootdowns
CPU226 CPU227 CPU0 CPU225 CPU118 summed
2026-04-23 03:41:10 LOC 58 66 138 146 157 565 Local timer interrupts
2026-04-23 03:41:10 RES 10 8 7 4 3 32 Rescheduling interrupts
2026-04-23 03:41:10 CAL 103 35 0 0 0 138 Function call interrupts
2026-04-23 03:41:10 4 0 0 2 0 0 2 IO-APIC-edge dma channel
2026-04-23 03:41:10 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:10 26 0 0 4 0 0 4 SBOX-ICR-edge vnet intr
2026-04-23 03:41:10 TLB 0 0 0 1 0 1 TLB shootdowns
CPU226 CPU227 CPU0 CPU225 CPU34 summed
2026-04-23 03:41:12 LOC 130 81 106 105 160 582 Local timer interrupts
2026-04-23 03:41:12 RES 15 10 8 3 2 38 Rescheduling interrupts
2026-04-23 03:41:12 CAL 0 58 0 22 0 80 Function call interrupts
2026-04-23 03:41:12 4 0 0 2 0 0 2 IO-APIC-edge dma channel
2026-04-23 03:41:12 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:12 26 0 0 4 0 0 4 SBOX-ICR-edge vnet intr
2026-04-23 03:41:12 TLB 0 0 0 1 0 1 TLB shootdowns
CPU226 CPU227 CPU0 CPU225 CPU4 summed
2026-04-23 03:41:13 LOC 139 76 91 88 158 552 Local timer interrupts
2026-04-23 03:41:13 RES 14 9 8 6 2 39 Rescheduling interrupts
2026-04-23 03:41:13 CAL 0 50 3 74 0 127 Function call interrupts
2026-04-23 03:41:13 4 0 0 2 0 0 2 IO-APIC-edge dma channel
2026-04-23 03:41:13 5 0 0 0 0 0 0 IO-APIC-edge dma channel
2026-04-23 03:41:13 26 0 0 4 0 0 4 SBOX-ICR-edge vnet intr
2026-04-23 03:41:13 TLB 0 0 1 2 0 3 TLB shootdowns
While the above command showed the interrupts activity on the MIC, the following command shows the interrupt activity on the host hpmc9. It is captured while an OMP program was running over the xphi.
# 03:43 root@HPMC9: /tools # ./dint -NVU 5 -iter 7 -delay 1 -interrupts LOC,RES,CAL,TLB -sortby RES
03:43 root@HPMC9: /tools # ./dint -NVU 5 -iter 7 -delay 1 -interrupts LOC,RES,CAL,TLB -sortby RES
CPU0 CPU5 CPU43 CPU37 CPU34 summed
2026-04-23 03:44:00 LOC 8 2 0 1 4 15 Local timer interrupts
2026-04-23 03:44:00 RES 6 0 0 0 0 6 Rescheduling interrupts
2026-04-23 03:44:00 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:00 TLB 0 0 0 0 0 0 TLB shootdowns
CPU0 CPU5 CPU43 CPU37 CPU34 summed
2026-04-23 03:44:01 LOC 4 2 3 3 0 12 Local timer interrupts
2026-04-23 03:44:01 RES 3 0 0 0 0 3 Rescheduling interrupts
2026-04-23 03:44:01 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:01 TLB 0 0 0 0 0 0 TLB shootdowns
CPU0 CPU30 CPU5 CPU43 CPU37 summed
2026-04-23 03:44:02 LOC 7 56 4 0 3 70 Local timer interrupts
2026-04-23 03:44:02 RES 2 1 0 0 0 3 Rescheduling interrupts
2026-04-23 03:44:02 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:02 TLB 0 0 0 0 0 0 TLB shootdowns
CPU0 CPU5 CPU43 CPU37 CPU34 summed
2026-04-23 03:44:03 LOC 5 2 2 2 3 14 Local timer interrupts
2026-04-23 03:44:03 RES 2 0 0 0 0 2 Rescheduling interrupts
2026-04-23 03:44:03 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:03 TLB 0 0 0 0 0 0 TLB shootdowns
CPU0 CPU5 CPU43 CPU37 CPU34 summed
2026-04-23 03:44:04 LOC 4 4 3 3 5 19 Local timer interrupts
2026-04-23 03:44:04 RES 5 0 0 0 0 5 Rescheduling interrupts
2026-04-23 03:44:04 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:04 TLB 0 0 0 0 0 0 TLB shootdowns
CPU0 CPU30 CPU5 CPU43 CPU37 summed
2026-04-23 03:44:05 LOC 10 52 5 0 0 67 Local timer interrupts
2026-04-23 03:44:05 RES 2 2 0 0 0 4 Rescheduling interrupts
2026-04-23 03:44:05 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:05 TLB 0 0 0 0 0 0 TLB shootdowns
CPU0 CPU5 CPU43 CPU37 CPU34 summed
2026-04-23 03:44:06 LOC 6 4 2 2 6 20 Local timer interrupts
2026-04-23 03:44:06 RES 3 0 0 0 0 3 Rescheduling interrupts
2026-04-23 03:44:06 CAL 0 0 0 0 0 0 Function call interrupts
2026-04-23 03:44:06 TLB 0 0 0 0 0 0 TLB shootdowns