There is no restriction on the number of contact lists or number of the contacts in a list.
You can upload as many as contacts you want to since there is no limit on a number of contact you have.
But definitely system has its own limit and capacity to upload any list at a particular time.
Your list size should be less than 20 MB file for a specific file during a single upload, although you can split a bigger list into the smaller lists and upload them one by one.
We can increase this uploading size limit from 20 MB to more higher number based your server capacity (RAM, Space, CPU etc), but it will be suggested to use the smaller list to import for better performance of the system.