Categories
DailyOps

How to solve netcat mystery X

Funny netcat issue with additional X characters send over UDP protocol and why this is not a problem.

The funny issue

Send something over network using UDP protocol.

$ netcat -q0 -u remote_address 2000 <<< "test message"

Receive it on the other side.

remote_address$ netcat -k -u -l 2000
test message

Everything seems fine, but use a verbose mode to see something not expected.

$ netcat -q0 -u remote_address 2000 -v <<< "test message"
Connection to remote_server (172.16.0.1) 2000 port [udp/*] succeeded!

Receive it on the other side.

remote_server$ netcat -k -u -l 2000
XXXXXtest message

Notice the additional X characters.

This is not a problem

This is not an issue with your network stack or an application itself.

The strace will clearly show that this data is sent deliberately.

$ strace netcat -q0 -u remote_server 2000 -v <<< "test message"
execve("/usr/bin/netcat", ["netcat", "-q0", "-u", "remote_server", "2000", "-v"], 0x7ffd64fa26e8 /* 31 vars */) = 0 
[...]
connect(3, {sa_family=AF_INET, sin_port=htons(2000), sin_addr=inet_addr("172.16.0.1")}, 16) = 0
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
write(3, "X", 1)                        = 1
write(3, "X", 1)                        = 1
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7ffcb539cff0) = 0
write(3, "X", 1)                        = 1
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7ffcb539cff0) = 0
write(3, "X", 1)                        = 1
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7ffcb539cff0) = 0
write(3, "X", 1)                        = 1
newfstatat(AT_FDCWD, "/etc/nsswitch.conf", {st_mode=S_IFREG|0644, st_size=510, ...}, 0) = 0
openat(AT_FDCWD, "/etc/services", O_RDONLY|O_CLOEXEC) = 4
newfstatat(4, "", {st_mode=S_IFREG|0644, st_size=12813, ...}, AT_EMPTY_PATH) = 0
lseek(4, 0, SEEK_SET)                   = 0
read(4, "# Network services, Internet sty"..., 4096) = 4096
read(4, "\ntinc\t\t655/tcp\t\t\t\t# tinc control"..., 4096) = 4096
read(4, "cd\t6445/tcp\tsge_execd\t# Grid Eng"..., 4096) = 4096
read(4, "es\namidxtape\t10083/tcp\t\t\t# amand"..., 4096) = 525
read(4, "", 4096)                       = 0
close(4)                                = 0
write(2, "Connection to 172.16.0.1", 24Connection to 172.16.0.1) = 24
write(2, " 2000 port [udp/*] succeeded!\n", 30 2000 port [udp/*] succeeded!
) = 30
poll([{fd=0, events=POLLIN}, {fd=3, events=0}, {fd=3, events=POLLIN}, {fd=1, events=0}], 4, -1) = 1 ([{fd=0, revents=POLLIN|POLLHUP}])
read(0, "test message\n", 16384)        = 13
poll([{fd=0, events=POLLIN}, {fd=3, events=POLLOUT}, {fd=3, events=POLLIN}, {fd=1, events=0}], 4, -1) = 2 ([{fd=0, revents=POLLHUP}, {fd=3, revents=POLLOUT}])
write(3, "test message\n", 13)          = 13
shutdown(3, SHUT_WR)                    = 0
shutdown(3, SHUT_RD)                    = 0
close(3)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

This is because I am using an OpenBSD version of netcat.

$ netcat -h
OpenBSD netcat (Debian patchlevel 1.217-3ubuntu1)
usage: nc [-46CDdFhklNnrStUuvZz] [-I length] [-i interval] [-M ttl]
          [-m minttl] [-O length] [-P proxy_username] [-p source_port]
          [-q seconds] [-s sourceaddr] [-T keyword] [-V rtable] [-W recvlimit]
          [-w timeout] [-X proxy_protocol] [-x proxy_address[:port]]
          [destination] [port]
        Command Summary:
                -4              Use IPv4
                -6              Use IPv6
                -b              Allow broadcast
                -C              Send CRLF as line-ending
                -D              Enable the debug socket option
                -d              Detach from stdin
                -F              Pass socket fd
                -h              This help text
                -I length       TCP receive buffer length
                -i interval     Delay interval for lines sent, ports scanned
                -k              Keep inbound sockets open for multiple connects
                -l              Listen mode, for inbound connects
                -M ttl          Outgoing TTL / Hop Limit
                -m minttl       Minimum incoming TTL / Hop Limit
                -N              Shutdown the network socket after EOF on stdin
                -n              Suppress name/port resolutions
                -O length       TCP send buffer length
                -P proxyuser    Username for proxy authentication
                -p port         Specify local port for remote connects
                -q secs         quit after EOF on stdin and delay of secs
                -r              Randomize remote ports
                -S              Enable the TCP MD5 signature option
                -s sourceaddr   Local source address
                -T keyword      TOS value
                -t              Answer TELNET negotiation
                -U              Use UNIX domain socket
                -u              UDP mode
                -V rtable       Specify alternate routing table
                -v              Verbose
                -W recvlimit    Terminate after receiving a number of packets
                -w timeout      Timeout for connects and final net reads
                -X proto        Proxy protocol: "4", "5" (SOCKS) or "connect"
                -x addr[:port]  Specify proxy address and port
                -Z              DCCP mode
                -z              Zero-I/O mode [used for scanning]
        Port numbers can be individual or ranges: lo-hi [inclusive]

Inspect source code of openbsd/netcat at GitHub to understand this behavior.

[...]
			if (vflag || zflag) {
				/* For UDP, make sure we are connected. */
				if (uflag) {
					if (udptest(s) == -1) {
						ret = 1;
						continue;
					}
				}
[...]
[...]
/*
 * udptest()
 * Do a few writes to see if the UDP port is there.
 * Fails once PF state table is full.
 */
int
udptest(int s)
{
	int i, ret;

	for (i = 0; i <= 3; i++) {
		if (write(s, "X", 1) == 1)
			ret = 1;
		else
			ret = -1;
	}
	return ret;
}
[...]

This behavior is expected in OpenBSD version of netcat when using a verbose mode.

An alternative

If this bothers you then you can use the traditional version of netcat.

$ apt search ^netcat
Sorting... Done
Full Text Search... Done
netcat/impish 1.217-3ubuntu1 all
  TCP/IP swiss army knife -- transitional package

netcat-openbsd/impish,now 1.217-3ubuntu1 amd64 [installed,automatic]
  TCP/IP swiss army knife

netcat-traditional/impish 1.10-47 amd64
  TCP/IP swiss army knife

netrw/impish 1.3.2-3 amd64
  netcat like tool with nice features to transport files over network

Install netcat-traditional package.

$ sudo apt install netcat-traditional

Display information about the nc link group.

$ update-alternatives --display nc
nc - auto mode
  link best version is /bin/nc.openbsd
  link currently points to /bin/nc.openbsd
  link nc is /bin/nc
  slave nc.1.gz is /usr/share/man/man1/nc.1.gz
  slave netcat is /bin/netcat
  slave netcat.1.gz is /usr/share/man/man1/netcat.1.gz
/bin/nc.openbsd - priority 50
  slave nc.1.gz: /usr/share/man/man1/nc_openbsd.1.gz
  slave netcat: /bin/nc.openbsd
  slave netcat.1.gz: /usr/share/man/man1/nc_openbsd.1.gz
/bin/nc.traditional - priority 10
  slave nc.1.gz: /usr/share/man/man1/nc.traditional.1.gz
  slave netcat: /bin/nc.traditional
  slave netcat.1.gz: /usr/share/man/man1/nc.traditional.1.gz

Set nc.traditional as alternative for nc.

$ sudo update-alternatives --set  nc /bin/nc.traditional
update-alternatives: using /bin/nc.traditional to provide /bin/nc (nc) in manual mode

Display updated settings.

$ update-alternatives --display nc
nc - manual mode
  link best version is /bin/nc.openbsd
  link currently points to /bin/nc.traditional
  link nc is /bin/nc
  slave nc.1.gz is /usr/share/man/man1/nc.1.gz
  slave netcat is /bin/netcat
  slave netcat.1.gz is /usr/share/man/man1/netcat.1.gz
/bin/nc.openbsd - priority 50
  slave nc.1.gz: /usr/share/man/man1/nc_openbsd.1.gz
  slave netcat: /bin/nc.openbsd
  slave netcat.1.gz: /usr/share/man/man1/nc_openbsd.1.gz
/bin/nc.traditional - priority 10
  slave nc.1.gz: /usr/share/man/man1/nc.traditional.1.gz
  slave netcat: /bin/nc.traditional
  slave netcat.1.gz: /usr/share/man/man1/nc.traditional.1.gz

Display netcat version and help information.

$ netcat -h
[v1.10-47]
connect to somewhere:   nc [-options] hostname port[s] [ports] ... 
listen for inbound:     nc -l -p port [-options] [hostname] [port]
options:
        -c shell commands       as `-e'; use /bin/sh to exec [dangerous!!]
        -e filename             program to exec after connect [dangerous!!]
        -b                      allow broadcasts
        -g gateway              source-routing hop point[s], up to 8
        -G num                  source-routing pointer: 4, 8, 12, ...
        -h                      this cruft
        -i secs                 delay interval for lines sent, ports scanned
        -k                      set keepalive option on socket
        -l                      listen mode, for inbound connects
        -n                      numeric-only IP addresses, no DNS
        -o file                 hex dump of traffic
        -p port                 local port number
        -r                      randomize local and remote ports
        -q secs                 quit after EOF on stdin and delay of secs
        -s addr                 local source address
        -T tos                  set Type Of Service
        -t                      answer TELNET negotiation
        -u                      UDP mode
        -v                      verbose [use twice to be more verbose]
        -w secs                 timeout for connects and final net reads
        -C                      Send CRLF as line-ending
        -z                      zero-I/O mode [used for scanning]
port numbers can be individual or ranges: lo-hi [inclusive];
hyphens in port names must be backslash escaped (e.g. 'ftp\-data').

Send something using verbose mode.

$ netcat  -vv -q0 -u remote_server 2000 -v <<< "test message"
remote_server [172.16.0.1] 2000 (?) open

Receive a message on remote server.

remote_server$ netcat -u -l -p 2000
test message

Although, this solution has its own drawbacks.