sc07 Thursday all about iWARP
Learned about iWARP today. This I think is the one major new things I learned at SC this year. We talked to two vendors about their nic’s and what iWARP was and how it worked.
iWARP looks like its just RDMA TCP. Uses standard ethernet switches (which is good for our Force10) CX4 and twisted pair but you need NIC’s that have the extra ability written into them. As for MPI library support iWARP is in OFED which OpenMPI supports and we use already to support our Infiniband. The really killer thing about this is the NIC will do this fancy RDMA over TCP and still do regular TCP/IP in the kernel when talking to other devices.
While latency does not look as good as Infiniband its sub 10 micro second which is great and in most cases good enough. Some vendors made claims also about maintaining latency for a large number of connections being made at once. I plan to test this my self with some MPI_Alltoall() tests on differnt number of processors. If the Infiniband shows the behavior claimed I will try to lean on my boss to request some demo hardware.
In the final point I really think the sweet point for iWARP would be 1Gbps cards that use twisted pair and keep the price low. I would gladly put one of these cards in every node i had and use it as my only interface if it was in the sub $500 range. This is much less than Infiniband because in Infiniband you have to buy a extra switch and use those damn CX4 cables ($#%^).
I Don’t know but I was impressed and if the price is right (which it DOES NOT appear to be so at first look) its a great low cost way to go. I would really like to hear what others have to say on this or if you have tried iWARP email me at: brockp@mlds-networks.com or feel free to comment below.
Brock
