HidSharp Read Timeout

I have been using HidSharp on a RPi and it is been running fine.
I allow up to 4 streams and poll each of the 4 devices/streams (if used and not null) every 50mS.
Up until now I have used with multiple Hid devices successfully. Mostly card reader devices (but with keyboards/mice that I ignore).
I now have a numeric keypad but every few times my app runs, it enumerates and identifies the device but no data is received until I restart the unit. The same keypad when using a text editor is working, just not on my reads in my app.
If polling, what Read Timeout would I use? The same or less than the polling rate, or set it high and do not worry?