Bug 14671

you can reproduce it like this:

# install kind: https://kind.sigs.k8s.io/docs/user/quick-start/
kind create cluster


guettli@p15:~/.kube$ cat busy-new.yaml 

apiVersion: v1
kind: Pod
metadata:
  name: busybox-new
  namespace: default
spec:
  containers:
  - name: busybox
    image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
  restartPolicy: Always

guettli@p15:~/.kube$ kubectl apply -f busy-new.yaml

guettli@p15:~/.kube$ kubectl exec -ti busybox-new -- nslookup kubernetes.default
Server:		10.96.0.10
Address:	10.96.0.10:53

** server can't find kubernetes.default: NXDOMAIN

----------

If you use "busybox:1.28" as image, it works fine.

A simpler reproducer (that does not require Kubernetes) is something like this:

$ docker run --rm --pull=always --dns-search=f1 --dns-search=f2 --dns-search=f3 --dns-search=google.com --dns 1.1.1.1 busybox:uclibc nslookup www
...
** server can't find www.f1: NXDOMAIN
...


If you drop the number of search domains down to just three, it works:

$ docker run --rm --pull=always --dns-search=f1 --dns-search=f2 --dns-search=google.com --dns 1.1.1.1 busybox:uclibc nslookup www
...
Non-authoritative answer:
Name:	www.google.com
Address: 142.250.176.4
...

This was on BusyBox version 1.34.1 built against uClibc-ng via buildroot.

Interestingly, it does *not* reproduce if I switch to "busybox:glibc" (built against Debian's glibc) or "busybox:musl" (built against Alpine's musl).

Can you confirm whether this is still related to concurrent issue of A and AAAA lookup as per https://bugs.busybox.net/show_bug.cgi?id=11161#c4 ?

For the prior bug, specifying -type=a consistently removed the intermittent failure (which presented when there was no AAAA record).

Just for the recors, in the past there have been DNS issues with other libc implementations. Maybe this helps to solve the uclibc-ng issue:

https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues