Spaces:
Sleeping
Sleeping
| Ticket Name: RTOS/TDA2SX: Network speed problem with 1000Mb/s connection | |
| Query Text: | |
| Part Number: TDA2SX Other Parts Discussed in Thread: X5777ATXGABC Tool/software: TI-RTOS Dear TI, We have a problem with network speed. We are using TDA2x EVM board and VISION_SDK_02_12_01_00 (RTOS, NDK_PROC_TO_USE=a15_0). When sending raw or encoded frames or just any data from the board to PC (using static IP and directly connected, NDK: Link Status: 1000Mb/s Full Duplex on PHY 7) bandwidth is about 21 Mbit/s. On PC side we are receiving frames with network_rx tool and our custom application. We have tried with Linux Ubuntu 16.04 18.04 and windows 10 but bandwidth is 21 Mbit/s. We have measured bandwidth using usecase and a simple module, which sends test data. When sending data from PC to board using network_tx tool bandwith is 110 Mbit/s. When sending data from one board to another board bandwidth is 220 Mbit/s. If we connect PC and the board with 100Mb/s Full Duplex then bandwith is about 90 Mbit/s. Do you have any suggestions, what can be an issue here with 1000Mb/s connection? Maybe some network configuration on PC? Best regards, Marko. | |
| Responses: | |
| Hi Marko, The VISION_SDK_02_12_01_00 version is quite old. The latest one is 03_06_00_00. Please try your use case on that and see if you're still able to see a bandwidth cap. This probably isn't the root cause, but you could try it. When running you application board to PC, did you check the network traffic using Task Manager or System Monitor? Regards, Anand | |
| Hi Anand, We will try with the latest version of vision SDK (03_06_00_00) and we will reply you. We use Wireshark, Task Manager, System Monitor and our custom application for check network traffic, bandwidth is about 21 Mbit/s and there is no other network traffic. PC is directly connected to the board. Regards, Marko. | |
| Hi Anand, We tested network bandwidth on the latest vision SDK version (03_06_00_00) and the results are the same. Do you have any more suggestions? Regards, Marko. | |
| Hi Marko, You had mentioned that there isn't much deterioration in the network bandwidth when you use 100 Mbps, how did you switch to 100 Mbps mode? Did you change the macInitCfg in the ndk_nsp_hooks.c file? Also, I am not aware if there are any specific configuration to be done in the PC to increase bandwidth. If the PC supports Gigabit ethernet, it should work out of the box. Anyways, I'll try to set the board up and run a simple app and see if I am seeing a drop this high in the bandwidth. If not, we could conclude it's something to do with the app. Regards, Anand | |
| Hi Anand, We switched to 100 Mbps speed on PC(receiver) side with the following command: "ethtool -s eth0 speed 100 duplex full". We did not change any of the mentioned files. Regards, Marko. | |
| Hi Marko, Ok, so I think that the EVM is configured is set for inband detection. So when you set 100 Mbps on your PC side, EVM detects it and does the config appropriately. Can you do this check - -> Once you get the IP Address displayed, confirm that your ping from the PC works -> Go to ti_components/networking/ndk_2_24_02_31/packages/ti/ndk/winapps -> You should binaries send.exe/send.out there. Run them like "./send.out <ip-addr of the evm>" -> After this open system monitor/task manager to see what is the bandwidth being used Let me know what BW could you observe in this case:) Regards, Anand | |
| Hi Anand, We checked BW and these are our results: 1. When we run send.exe on PC we get BW around 330 Mbps: 2. When we run recv.exe on PC we get BW around 7.8 Mbps: Regards, Marko. | |
| Hi Marko, I think that then the delay might be something related to the application. Because in the best case we get around 460 Mbps, considering all the wire, sw delays and the like. Send showing 330 Mbps means that there is no config error as such from the sdk or driver point of view. Is there any other possible delay in the application? Regards, Anand | |
| Hi Anand, Can you please tell us what bandwidth you get when you send data from a board to the PC? In our test application, which sends data from a board to the PC, we do not have any delays. Just the network write function is executing in the while loop. Also, there are no delays in our receiving test application. If it will be easier for you, we can send you our test application source code, but we will need your email address. Regards, Marko. | |
| Hi Marko, I get around 330 Mbps on send. Since you mentioned that in 100 Mbps mode you are getting close to 90 Mbps BW, but BW drops drastically when you switch to 1000 Mbps. So the problem is only with 1000 Mbps mode right? Then I don't think it might be an application issue. As you said if you're only doing network_write, then it shouldn't matter. This is rather strange. BTW one thing, I am not sure which version of the TDA2XX are you using. If I remember right, some of them don't support 1000 Mbps. I'll try doing network_write in a loop like you said and see if I face a similar issue. Regards, Anand | |
| Hi Anand, Yes, the problem is only with 1000 Mbps mode. The board that we are using is VAYU EVM XC5777X CPU BOARD (ASSY 516580 REV). On Chip: -------------------------- OMAP X5777ATXGABC 54ZCMS9 842 ABC G1 -------------------------- Besides this board, we are using ALPHA AMV board which is our product. It is also using tda2XX chips. On RTOS behavior of the board is the same, bandwidth drops when we swtich from 100Mbps to 1Gbps Ethernet. When using Linux everything seems to be working, bandwidth is ~ 990 Mbps. Regards, Marko. | |
| Hi Marko, Ok I will check with the linux team to see if they are configuring the IODelays any differently. Since it works on VSDK linux for you, I believe it has got nothing to do with your PC setup. Regards, Anand | |
| Marko, My best guess is that since you have a different PHY on board, there is some fishy going on with the IO Delay configurations. In the ndk_nsp_hooks.c file inside utils_common/src/ndk folder, there is a function called NDK_NSP_init(). Please see to it that the delay configurations take place properly and you macConnectionType is rightly set up as MAC_CONNECTION_TYPE_RGMII_FORCE_1000_FULL or MAC_CONNECTION_TYPE_RGMII_DETECT_INBAND in case your on-board PHY suppports in band detection. Best Regards, Anand | |
| Hi Anand, We checked configurarion for macConnectionType and it is rightly set up as MAC_CONNECTION_TYPE_RGMII_FORCE_1000_FULL. Can you send us your Appimage and MLO with simple module for testing bandwidth? Also we can send you our Appimage and MLO to try it with your board. Best Regards, Marko. | |
| Marko, In case of Linux, the stack, drivers etc for networking are different. In the case of RTOS, we have NDK to provide the stack and NSP as device driver. This might explain the BW difference you see in both these cases. But still, I don't quite understand why the speeds would go so low. Just for a final clarification, once the SDK boots up, and you're able to ping to the IP, can you go and read the register address 0x48485288 and tell me the value? Regards, Anand | |
| Hi Anand, We read the register address 0x48485288 with HW_RD_REG32(). Value of this register is 13. Best Regards, Marko. | |
| Marko, This is infact the expected value. I agree, there is a little bit of throughput reduction in RTOS because of usage of a different stack etc. But in any case it should not be as low as it is in your case. Do you see the same trend when NDK runs on M4 also? Regards, Anand | |
| Hi Marko, Did you get a chance to try it on M4 also? Regards, Anand | |
| Hi Anand, Sorry for the late. We are out of office this week, then we will try on M4 next week and send you our results. Best regards, Marko. | |
| Hi Anand, We set network to M4 ( NDK_PROC_TO_USE=ipu1_0, Link Status: 1000Mb/s Full Duplex). We have bandwith about 45 Mbit/s (We Measured bandwith with system monitor and with our app). Best Regards, Marko. | |
| Hi Marko, That is actually lesser than what you should get ideally. Are you using any adapters of some sort in between? Regards, Anand | |
| Hi Anand, We do not use any adapter between board and PC. Board and PC are connected directly. Best Regards, Marko. | |
| Hi Marko, It is quite unfortunate that this issue is getting dragged, but the conclusions we see keep contradicting. You see almost 45 Mbps when NDK runs on M4, but nearly half of that when your run NDK in A15? I am not sure why that would happen. Let's try to look at it in a different perspective. I think we have a lot of variables here. What is the exact usecase you're trying to accomplish here? Regards, Anand | |
| Hi Anand, With this exact usecase, we want to stream from board to PC via Ethernet with 300 Mbps. Best Regards, Marko. | |
| Marko, In that case I suspect it has got something to do with the board. Because I'm not observing a speed deterioration in A15, that too lower than M4! I can try using your AppImage and MLO at my side to have complete closure. Also send in any changes you have made in ndk_nsp_hooks.c file, if any. Regards, Anand | |
| Hi Anand, How do you want us to send you our Appimage and MLO? Do you want them on email or private message on TI forum? Best Regards, Marko. | |
| Hi Marko, Are you not able to attach the files here? Also mention which app of yours are you trying to run. Regards, Anand | |
| Hi Anand, We want you to check bandwidth in two cases. First case is AppImage_MLO folder(Read note in folder). We sent you our AppImage and MLO and our test app. Second case is source folder(Read note in folder). We want that you add source files in your Vision SDK and build. When you build the test modules, test them the same way as previous case. https://dl.rt-rk.com/?t=f3c71be0c703aec79cd2029ff00fa4de Best Regards, Marko. | |
| Marko, Thank you for the files. I'll test this and get back to you soon. Regards, Anand | |
| Hi Marko, I was having some setup issues at my place (I don't currently work on this SoC family regularly). I will be able to test and tell you what's probably going wrong by end of this week. Regards, Anand | |
| Hi Marko, I don't observe the deterioration worsened for A15, compared to M4, so maybe re-check the setup once more to confirm on that. Anyways I see similar drops in BW. Network API uses TCP/IP. Have you tried UDP? I understand your application is streaming. Regards, Anand | |
| Hi Anand, Do you have bandwidth drops for two cases (with our appimage and MLO and with yours appimage and MLO i.e. when you build send task module)? We have some problems when testing with UDP but we hope to resolve this as soon as posible. Best Regards, Marko. | |
| Hi Marko, Both the drops are similar. I'll debug at my end and get back to you. Regards, Anand | |
| Hi Anand, We can not connect board and PC via UDP. But, 3 month ago we did it and we think that we had bandwidth drops. Best Regards, Marko. | |
| Hi Marko, I'm still debugging this issue. Let me know the priority on this. Do you have another board to test on? Regards, Anand | |
| Hi Anand, Priority on this is not high for now, because we moved to the another project. We tested this on multiple alpha boards (TDA2X) and on TDA2x EVM. Best Regards, Marko. | |
| Hi Marko, Thanks for the reply. I can continue the debug in parallel with the other things on my plate. Regards, Anand | |
| Hi Marko, I have not found any leads on this. Mostly it is a board issue. I will debug further for a week, and if nothing turns up, let's close this thread as issue is reprioritized. You can always open a new thread when it re surfaces. Regards, Anand | |
| Hi Marko, One factor which might improve the bandwidth is to turn off the dual mac mode and try this again. Dual mac mode did introduce some additional packet delays due to the switching logic. This involves changes in both SDK and the NSP, so I would suggest to go to VSDK 3.5, which doesn't have dual mac mode enabled. I would also suggest to do this change when the issue is reprioritized. You can open a new thread then to continue the discussion. We can close this thread for the time being. Regards, Anand | |
| Hi Anand, Thank you for the response. We will try this when the issue is reprioritized. Best Regards, Marko. | |
| Hi Marko, Can we close this thread in that case? | |
| Hi Anand, Yes, you can close this thread. We will open new thread if we have problem when dual mac is enabled. Best Regards, Marko. | |
| Hi Marko, Thanks for your reply. Regards, Anand | |