openssh 7.5p1だとIPv6でつながらない(その4: path mtu discovery問題)

openssh で自宅から接続できたりできなかったりする件ですが、path mtu discovery問題のような気がしてきました。
conoha VPSから ping6 -s SIZE targetすると、SIZEが小さければ到達できるけど、大きいと駄目。そもそもIPv6では途中のルータがfragment/reassembleできない筈なんだけど、自宅のcarrotへping6すると、フラグメント化して送ったのに届いた時は非フラグメント化されている。自宅からのパケットも同じ。これはconoha VPSの上位ルータで勝手に非フラグメント化させているような気がする。

(1) peach(conoha VPS) -> carrot(自宅)
peach# ping6 -s 1406 -c 3 carrot
PING6(1454=40+8+1406 bytes) 2400:8500:1302:815:163:44:174:253 –> 2400:7800::XXXX
1414 bytes from 2400:7800::XXXX, icmp_seq=0 hlim=47 time=9.491 ms
1414 bytes from 2400:7800::XXXX, icmp_seq=1 hlim=47 time=8.750 ms
1414 bytes from 2400:7800::XXXX, icmp_seq=2 hlim=47 time=5.011 ms
— carrot.ish.org ping6 statistics —
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.011/7.751/9.491/1.961 ms

peach# tcpdump icmp6 and host peach.ish.org and host carrot.ish.org
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:23:18.350810 IP6 peach > carrot: frag (0|1232) ICMP6, echo request, seq 0, length 1232
21:23:18.350823 IP6 peach > carrot: frag (1232|182) <— (1-1)peachで送った際はフラグメント化されている
21:23:18.360228 IP6 carrot > peach: ICMP6, echo reply, seq 0, length 1414
21:23:19.380647 IP6 peach > carrot: frag (0|1232) ICMP6, echo request, seq 1, length 1232
21:23:19.380660 IP6 peach > carrot: frag (1232|182)
21:23:19.389303 IP6 carrot > peach: ICMP6, echo reply, seq 1, length 1414
21:23:20.440677 IP6 peach > carrot: frag (0|1232) ICMP6, echo request, seq 2, length 1232
21:23:20.440690 IP6 peach > carrot: frag (1232|182)
21:23:20.445525 IP6 carrot > peach: ICMP6, echo reply, seq 2, length 1414

carrot# tcpdump icmp6 and host carrot.ish.org and peach.ish.org
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ue0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:23:18.674079 IP6 peach > carrot: ICMP6, echo request, seq 0, length 1414 <— (1-2)carrotで受信時はフラグメント化されていない!!!
21:23:18.674163 IP6 carrot > peach: ICMP6, echo reply, seq 0, length 1414
21:23:19.706515 IP6 peach > carrot: ICMP6, echo request, seq 1, length 1414
21:23:19.706599 IP6 carrot > peach: ICMP6, echo reply, seq 1, length 1414
21:23:20.762908 IP6 peach > carrot: ICMP6, echo request, seq 2, length 1414
21:23:20.762995 IP6 carrot > peach: ICMP6, echo reply, seq 2, length 1414

(2) carrot -> peach
carrot# ping6 -s 1400 -c 3 peach.ish.org
PING6(1448=40+8+1400 bytes) 2400:7800::XXXX –> 2400:8500:1302:815:163:44:174:253
1408 bytes from 2400:8500:1302:815:163:44:174:253, icmp_seq=0 hlim=50 time=27.359 ms
1408 bytes from 2400:8500:1302:815:163:44:174:253, icmp_seq=1 hlim=50 time=5.936 ms
1408 bytes from 2400:8500:1302:815:163:44:174:253, icmp_seq=2 hlim=50 time=6.611 ms
— peach.ish.org ping6 statistics —
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.936/13.302/27.359/9.944 ms

carrot# tcpdump icmp6 and host carrot.ish.org and peach.ish.org
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ue0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:30:05.096541 IP6 carrot > peach: frag (0|1232) ICMP6, echo request, seq 0, length 1232
21:30:05.096547 IP6 carrot > peach: frag (1232|176) <— (2-1)carrotからフラグメント化したecho requestを送信
21:30:05.123792 IP6 peach > carrot: ICMP6, echo reply, seq 0, length 1408 <— (2-4)carrotへ届いたecho reply はフラグメント化されていない!!!
21:30:06.108269 IP6 carrot > peach: frag (0|1232) ICMP6, echo request, seq 1, length 1232
21:30:06.108274 IP6 carrot > peach: frag (1232|176)
21:30:06.114064 IP6 peach > carrot: ICMP6, echo reply, seq 1, length 1408
21:30:07.109867 IP6 carrot > peach: frag (0|1232) ICMP6, echo request, seq 2, length 1232
21:30:07.109873 IP6 carrot > peach: frag (1232|176)
21:30:07.116378 IP6 peach > carrot: ICMP6, echo reply, seq 2, length 1408

peach# tcpdump icmp6 and host peach.ish.org and host carrot.ish.org
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:30:04.788922 IP6 carrot > peach: ICMP6, echo request, seq 0, length 1408 <— (2-2)peachで受信したecho requestはフラグメント化されていない!!!
21:30:04.788963 IP6 peach > carrot: frag (0|1352) ICMP6, echo reply, seq 0, length 1352
21:30:04.788973 IP6 peach > carrot: frag (1352|56) <— (2-3)peachからのecho replyはフラグメント化して送信
21:30:05.801132 IP6 carrot > peach: ICMP6, echo request, seq 1, length 1408
21:30:05.801180 IP6 peach > carrot: frag (0|1352) ICMP6, echo reply, seq 1, length 1352
21:30:05.801213 IP6 peach > carrot: frag (1352|56)
21:30:06.802608 IP6 carrot > peach: ICMP6, echo request, seq 2, length 1408
21:30:06.802641 IP6 peach > carrot: frag (0|1352) ICMP6, echo reply, seq 2, length 1352
21:30:06.802652 IP6 peach > carrot: frag (1352|56)

(3) 自宅ではちゃんと3000バイトのpingがping-tokyo.sinet.ad.jpへ届く
carrot# ping6 -s 3000 -c 3 ping-tokyo.sinet.ad.jp
PING6(3048=40+8+3000 bytes) 2400:7800::XXXX –> 2001:2f8:fd0e::3
3008 bytes from 2001:2f8:fd0e::3, icmp_seq=0 hlim=56 time=14.436 ms
3008 bytes from 2001:2f8:fd0e::3, icmp_seq=1 hlim=56 time=9.897 ms
3008 bytes from 2001:2f8:fd0e::3, icmp_seq=2 hlim=56 time=36.159 ms
— ping-tokyo.sinet.ad.jp ping6 statistics —
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 9.897/20.164/36.159/11.461 ms

carrot# tcpdump icmp6 and host carrot.ish.org and ping-tokyo.sinet.ad.jp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ue0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:36:51.834155 IP6 carrot > 2001:2f8:fd0e::3: frag (0|1232) ICMP6, echo request, seq 0, length 1232
21:36:51.834172 IP6 carrot > 2001:2f8:fd0e::3: frag (1232|1232)
21:36:51.834180 IP6 carrot > 2001:2f8:fd0e::3: frag (2464|544)
21:36:51.848049 IP6 2001:2f8:fd0e::3 > carrot: frag (0|1400) ICMP6, echo reply, seq 0, length 1400
21:36:51.848286 IP6 2001:2f8:fd0e::3 > carrot: frag (1400|1400)
21:36:51.848317 IP6 2001:2f8:fd0e::3 > carrot: frag (2800|208)
21:36:52.834941 IP6 carrot > 2001:2f8:fd0e::3: frag (0|1232) ICMP6, echo request, seq 1, length 1232
21:36:52.834959 IP6 carrot > 2001:2f8:fd0e::3: frag (1232|1232)
21:36:52.834964 IP6 carrot > 2001:2f8:fd0e::3: frag (2464|544)
21:36:52.844303 IP6 2001:2f8:fd0e::3 > carrot: frag (0|1400) ICMP6, echo reply, seq 1, length 1400
21:36:52.844516 IP6 2001:2f8:fd0e::3 > carrot: frag (1400|1400)
21:36:52.844562 IP6 2001:2f8:fd0e::3 > carrot: frag (2800|208)
21:36:53.849616 IP6 carrot > 2001:2f8:fd0e::3: frag (0|1232) ICMP6, echo request, seq 2, length 1232
21:36:53.849632 IP6 carrot > 2001:2f8:fd0e::3: frag (1232|1232)
21:36:53.849639 IP6 carrot > 2001:2f8:fd0e::3: frag (2464|544)
21:36:53.885282 IP6 2001:2f8:fd0e::3 > carrot: frag (0|1400) ICMP6, echo reply, seq 2, length 1400
21:36:53.885476 IP6 2001:2f8:fd0e::3 > carrot: frag (1400|1400)
21:36:53.885526 IP6 2001:2f8:fd0e::3 > carrot: frag (2800|208)

(4) conoha VPSでは当然だめ
peach# ping6 -s 3000 -c 3 ping-tokyo.sinet.ad.jp
PING6(3048=40+8+3000 bytes) 2400:8500:1302:815:163:44:174:253 –> 2001:2f8:fd0e::3
— ping-tokyo.sinet.ad.jp ping6 statistics —
3 packets transmitted, 0 packets received, 100.0% packet loss

peach# tcpdump icmp6 and host peach.ish.org and host ping-tokyo.sinet.ad.jp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:39:31.013065 IP6 peach > 2001:2f8:fd0e::3: frag (0|1232) ICMP6, echo request, seq 0, length 1232
21:39:31.013078 IP6 peach > 2001:2f8:fd0e::3: frag (1232|1232)
21:39:31.013083 IP6 peach > 2001:2f8:fd0e::3: frag (2464|544)
21:39:32.070541 IP6 peach > 2001:2f8:fd0e::3: frag (0|1232) ICMP6, echo request, seq 1, length 1232
21:39:32.070555 IP6 peach > 2001:2f8:fd0e::3: frag (1232|1232)
21:39:32.070560 IP6 peach > 2001:2f8:fd0e::3: frag (2464|544)
21:39:33.130554 IP6 peach > 2001:2f8:fd0e::3: frag (0|1232) ICMP6, echo request, seq 2, length 1232
21:39:33.130568 IP6 peach > 2001:2f8:fd0e::3: frag (1232|1232)
21:39:33.130573 IP6 peach > 2001:2f8:fd0e::3: frag (2464|544)

うーん、困ったなぁ。上位ルータが勝手に非フラグメント化するんじゃ手の施しようがない。

その3へ戻る  conoha VPSのpath MTU discovery blackhole問題(その1)へ続く