Thursday, August 22, 2024

A Practical Improvement in DNS Transport over UDP over IPv6

By Hugo Salgado and Alejandro Acosta


Introduction and problem statement

In this document we want to discuss an existing IETF draft (a working document that may become a standard) that caught our attention. This draft involves two fascinating universes: IPv6 and DNS. It introduces some best practices for carrying DNS over IPv6.


Its title is “DNS over IPv6 Best Practices” and it can be found here.


What is the document about and what problem does it seek to solve?

The document describes an approach to how Domain Name Protocol (DNS) should be carried over IPv6 [RFC8200].

Some operational issues have been identified in carrying DNS packets over IPv6 and the draft seeks to address them.


Technical context

The IPv6 protocol requires a minimum link MTU of 1280 octets. According to Section 5 “Packet Size Issues” of RFC8200, every link in the Internet must have an MTU of 1280 octets or greater. If a link cannot convey a 1280-octet packet in one piece, link-specific fragmentation and reassembly must be provided at a layer below IPv6.


Successful operation of PMTUD in an example adapted to 1280-byte MTU

Image source: https://www.slideshare.net/slideshow/naveguemos-por-internet-con-ipv6/34651833#2


Using Path MTU Discovery (PMTUD) and IPv6 fragmentation (source only) allows larger packets to be sent. However, operational experience shows that sending large DNS packets over UDP over IPv6 results in high loss rates. Some studies —quite a few years old but useful for context— found that around 10% of IPv6 routers drop all IPv6 fragments, and 40% block “Packet Too Big” messages, making client negotiation impossible. (“M. de Boer, J. Bosma, “Discovering Path MTU black holes on the Internet using RIPE Atlas”)

Most modern transport protocols like TCP [TCP] and QUIC [QUIC] include packet segmentation techniques that allow them to send larger data streams over IPv6.


A bit of history

The Domain Name System (DNS) was originally defined in RFC1034 and RFC1035. It was designed to run over several different transport protocols, including UDP and TCP, and has more recently been extended to run over QUIC. These transport protocols can be run over both IPv4 and IPv6.

When DNS was designed, the size of DNS packets carried over UDP was limited to 512 bytes. If a message was longer than 512 bytes, it was truncated and the Truncation (TC) bit was set to indicate that the response was incomplete, allowing the client to retry with TCP.

With this original behavior, UDP over IPv6 did not exceed the IPv6 link MTU (maximum transmission unit), avoiding operational issues due to fragmentation. However, with the introduction of EDNS0 extensions (RFC6891), the maximum was extended to a theoretical 64KB. This caused some responses to exceed the 512-byte limit for UDP, which resulted in sizes that exceeded the Path MTU and triggered TCP connections, impacting the scalability of the DNS servers.


Encapsulating a DNS packet in an Ethernet frame


Let’s talk about DNS over IPv6

DNS over IPv6 is designed to run over UDP or other transport protocols like TCP or QUIC. UDP only provides for source and destination ports, a length field, and simple checksum. It is a connectionless protocol. In contrast, TCP and QUIC offer additional features such as packet segmentation, reliability, error correction, and connection state.

DNS over UDP over IPv6 is suitable for small packet sizes, but becomes less reliable with larger sizes, particularly when IPv6 datagram fragmentation is required.

On the other hand, DNS over TCP or QUIC over IPv6 work well with all packet sizes. However, running a stateful protocol such as TCP or QUIC places greater demands on the DNS server’s resources (and other equipment such as firewalls, DPIs, and IDS), which can potentially impact scalability. This may be a reasonable tradeoff for servers that need to send larger DNS response packets.

The draft’s suggestion for DNS over UDP recommends limiting the size of DNS over UDP packets over IPv6 to 1280 octets. This avoids the need for IPv6 fragmentation or Path MTU Discovery, which ensures greater reliability.

Most DNS queries and responses will fit within this packet size limit and can therefore be sent over UDP. Larger DNS packets should not be sent over UDP; instead, they should be sent over TCP or QUIC, as described in the next section.


DNS over TCP and QUIC

When larger DNS packets need to be carried, it is recommended to run DNS over TCP or QUIC. These protocols handle segmentation and reliably adjust their segment size for different link and path MTU values, which makes them much more reliable than using UDP with IPv6 fragmentation.

Section 4.2.2 of [RFC1035] describes the use of TCP for carrying DNS messages, while [RFC9250] explains how to implement DNS over QUIC to provide transport confidentiality. Additionally, operation requirements for DNS over TCP are described in [RFC9210].


Security

Switching from UDP to TCP/QUIC for large responses means that the DNS server must maintain an additional state for each query received over TCP/QUIC. This will consume additional resources on the servers and affect the scalability of the DNS system. This situation may also leave the servers vulnerable to Denial of Service (DoS) attacks.


Is this the correct solution?

While we believe this solution will bring many benefits to the IPv6 and DNS ecosystem, it is a temporary operational fix and does not solve the root problem.

We believe the correct solution is ensuring that source fragmentation works, that PMTUD is not broken along the way, and that security devices allow fragmentation headers. This requires changes across various Internet actors, which may take a long time, but that doesn’t mean that we should abandon our efforts or stop educating others about the importance of doing the right thing.


Sunday, June 9, 2024

Cisco hidden command: bgp bestpath as-path multipath-relax

Hidden command

  bgp bestpath as-path multipath-relax


What for is this?

By default, Cisco does not do load-balance or distribute traffic between different ASs, this command allows it. Important, you must also use the maximum-paths command


Example:

router bgp 65001

 bgp router-id 1.1.1.1

 bgp log-neighbor-changes

 bgp bestpath as-path multipath-relax

 neighbor 2001:DB8:12::2 remote-as 65002

 neighbor 2001:DB8:12:10::2 remote-as 65002

 neighbor 2001:DB8:13:11::3 remote-as 65003

 !

 address-family ipv4

 no neighbor 2001:DB8:12::2 activate

 no neighbor 2001:DB8:12:10::2 activate

 no neighbor 2001:DB8:13:11::3 activate

 exit-address-family

 !

 address-family ipv6

 maximum-paths 3

 neighbor 2001:DB8:12::2 activate

 neighbor 2001:DB8:12:10::2 activate

 neighbor 2001:DB8:13:11::3 activate

 exit-address-family


Output after implementation:

     Network          Next Hop            Metric LocPrf Weight Path

 *m  2001:DB8::4/128  2001:DB8:12:10::2

                                                              0 65002 65004 ?

 *>                   2001:DB8:12::2                         0 65002 65004 ?

 *m                   2001:DB8:13:11::3

                                                              0 65003 65004 ?

 *m  2001:DB8:24:11::/64

                       2001:DB8:12:10::2

                                                              0 65002 65004 ?

 *>                   2001:DB8:12::2                         0 65002 65004 ?

 *m                   2001:DB8:13:11::3

                                                              0 65003 65004 ?

 *m  2001:DB8:34::/64 2001:DB8:12:10::2

                                                              0 65002 65004 ?

 *>                   2001:DB8:12::2                         0 65002 65004 ?

 *m                   2001:DB8:13:11::3

                                                              0 65003 65004 ?

Friday, June 7, 2024

Video: IPv6 LAC Race - May 2014 - Jun 2024

Do you want to know how the evolution of IPv6 has been in LAC? In this video of just a minute you will have your answer #barchartrace #ipv6





Sunday, June 2, 2024

Solved: "The following security updates require Ubuntu Pro with 'esm-apps' enabled"

Situation

  When you want to do some operations in Ubuntu using apt/do-release-upgrade you receive the message:

"The following security updates require Ubuntu Pro with 'esm-apps' enabled"


Solution

 The solution that worked for me was to run this:


cd /etc/apt/sources.list.d

for i in *.list; do mv ${i} ${i}.disabled; donated

apt clean

apt autoclean

sudo do-release-upgrade



Reference

https://askubuntu.com/questions/1085295/error-while-trying-to-upgrade-from-ubuntu-18-04-to-18-10-please-install-all-av 



Monday, April 29, 2024

Solved: Error: eth0 interface name is not allowed for R2 node when network mode is not set to none in containerlab

 Problem:

   Containerlab returns a similar error:

Error: eth0 interface name is not allowed for R2 node when network mode is not set to none


Solution:

 In the .yml file in the node section indicating the topology error specify:


network-mode: none


Example:

topology:

  kinds: 

    linux:

      image: quay.io/frrouting/frr:8.4.1

  nodes:

    R1:

      kind: linux

      image: quay.io/frrouting/frr:8.4.1

      network-mode: none


 Rerun the topology with clab dep -t file.yml and that's it!


Luck.

Friday, March 8, 2024

BGP Stream: An Analysis of One Year of BGP Incidents

04/03/2024


By Alejandro Acosta, R&D Coordinator at LACNIC

LACNIC presents the first webpage designed to show incidents and an analysis of Border Gateway Protocol (BGP) measurement data in Latin America and the Caribbean.

MAIN INCIDENTS. In addition to a summary of the information, the page shows three main types of events: possible network hijacks, BGP outages, and route leaks.

Possible hijacks refers to the illegitimate takeover of groups of IP addresses by corrupting Internet routing tables. This typically occurs when an Autonomous System announces a prefix that it does not originate.

Outages refers to the loss of visibility of network prefixes by a majority group of sensors.

Route leaks, as the name suggests, refers to the —potentially— unintentional announcement of a network prefix via BGP. For example, in a private peering traffic exchange, when one of the participants announces the peer’s prefix to the Internet. This case is the most difficult for algorithms to detect, so some of these incidents are not identified.

How is the data obtained?

This initiative uses Cisco BGP Stream, an automated process that selects the largest and most important incidents, providing information on the nature of the event and the ASNs involved.

The information is openly published, as LACNIC believes that it is important for engineers, network administrators, and organizations to gain insights into the most common incidents in the region and raise awareness about the situation.

This allows quickly investigating events, the rapid development of complex prototypes and tools, as well as large-scale monitoring applications (e.g., detecting connectivity outages, attacks, or BGP hijacks).

Using a system developed by LACNIC’s R&D department, raw data is collected, plotted, identified, cleaned, stored in a database, and later used to produce statistics and graphs. This occurs automatically every 24 hours.

RESULTS. During the study period —February 2023 to February 2024— we found the results shown in the charts below, which compare BGP events worldwide vs BGP events in our region.

A comparison between the global chart and the chart specific to the LAC region shows a similar pattern in the order of the most common incidents, with outages being the most frequent type of incident, followed by possible hijacks, and finally prefix leaks. It should also be noted that outages represent a higher percentage of the total number of incidents in our region than at the global level.

An analysis of the results table showing worldwide BGP events vs BGP events in our region reveals the following:

TOP 5 countries in our region with the highest number of BGP outages

Outages 
CCEvents
BR781
AR99
HT24
MX22
CL17

TOP 5 countries in our region with the highest number of possible Hijacks

Expected CCDetected CCEvents
BRBR67
BRnone35
PYBR24
BRUS22
BRCN9

TOP 3 countries in our region with the highest number of route leaks

Origin CCLeaker CCEvents
VEVE7
MXMX5
CLPA2

Impact

In this first year of operation, LACNIC has observed a reduction in BGP incidents. Several reasons for this have been identified, including a) the deployment and adoption of Resource Certification (RPKI), b) LACNIC’s Internet Routing Registry (IRR), and the adoption of RFC 9234 (Route Leak Prevention and Detection Using Roles in UPDATE and OPEN Messages).

The adoption of these tools is being driven by better operator practices and ISOC’s promotion of MANRS.

Conclusions

Possible hijacks, outages, and route leaks are the most common types of BGP incidents. During the initial year of data collection, a decrease in the number of cases was observed. However, it is expected that they will not disappear entirely in the near future. Implementing robust redundancy and resiliency measures in networks is crucial, as is the early detection and prevention of possible hijacks to ensure the integrity and reliability of Internet routes.

At LACNIC, our goal is to raise awareness and encourage ISPs and organizations to be prepared to handle these incidents efficiently when they occur.

References

https://stats.labs.lacnic.net/BGP/bgpstream-lac-region.html

https://stats.labs.lacnic.net/BGP/bgpstream.html

https://bgpstream.crosswork.cisco.com/ 


Tuesday, February 27, 2024

This is the way to install the telnet command in Alpine Linux (very popular in the container world such as docker)

This is the way to install the telnet command in Alpine Linux (very popular in the container world such as docker)

#apk update

#apk add busybox-extras