TUN interfaces are not created after creating and removing UEs and subscribers multiple times

Q/A
#1

Hi,

My setup is Setup:

  • OS: Ubuntu 20.04
  • Kernel: 5.4.0-147-generic
  • 32 cores
  • 64GB memory
  • free5gc-compose: commit hash f726cdd2c80c18a4fcce33fcfcd007844d39e74d (recent)

To reproduce the results do:

  1. Remove the containers and create them again to ensure testing on a clean slate.
  2. Create 150 subscribers
  3. Create 150 UEs with there individual process, i.e without the -n flag
  4. Test the connection
  5. Remove all UEs
  6. Remove all subscribers
  7. Create 150 subscribers
  8. Create 150 UEs with there individual process, i.e without the -n flag
  9. Test the connection (FAILS)

How I reproduce it:
I have written a script that creates 150 UEs and Subscribers. Each UE is created with
docker exec -d ueransim sh -c "./nr-ue -c ./config/uecfg.yaml -i $1 > devices.logs"
where $1 is the imsi, i.e., I create imsi-208930000000001 to imsi-208930000000150.
Each subscriber is created with curl -v -X POST $api/subscriber/$1/20893 -H "Token: admin" -d @$subscriber_conf where $1 is the imsi.
Then I can see that all 150 TUN interfaces are created when running docker exec ueransim ip link show. Each connection is then tested with
docker exec ueransim ping -c 1 -I $tun google.com >/dev/null 2>&1,
$tun starts from uesimtun0 and goes to uesimtun149. All of the connections are successful. I then remove all UEs and Subscribers, which is done with sudo kill -SIGINT $UE_PROCESS and curl -v -X DELETE $api/subscriber/$1/20893 -H "Token: admin".
When running docker exec ueransim sh -c "./nr-cli --dump" I can see that only UERANSIM-gnb-208-93-1 is there. Meaning that all of the UEs are removed. In webconsole I can also see that there is no subscribers.

The problem is that if I now try to create 150 UEs and subscribers again I can’t.
Only a few TUN interfaces are created.

However, if I do the same process but after deleting all UEs and subscribers I restart the containers it works, i.e., with stop/start not down/up.

Thank you for your help.

#2

Hi @Anton-Augustsson,

Thanks for providing very detailed steps for reproducing the problem.
I found there are two issues:

  1. The default configuration may not suit your test case.
  2. If you create multiple UEs too fast, the ueransim will be too busy to create the TUN interface.

Here is a temporary solution for you:

  1. Change cidr of dnnList in the upfcfg.yaml. e.g. 10.60.0.0/24 to 10.60.0.0/16
  2. After you create a UE, sleep for 10 milliseconds.

After doing these, I can pass step 9 in your reproducing procedure.

Thank you!!

#3

Hi @Johnson3310,

Thank you for your answer. Changing the subnet mask for the UPF solves the issue. Not sure when or if it is possible to reuse IP addresses instead. However, we will not scale it so much for our case that we have to use more than 65536 addresses (available addresses for \16).

#4