- Release notes
- Overview
- Setup and configuration
- Software requirements
- Hardware requirements
- Deploying the server
- Connecting to the server
- Licensing
- Data storage
AI Computer Vision user guide
This setup works on on-premises Nvidia GPUs, but also works with cloud providers such as AWS, Azure and GCP. Suggested GPU types include those from the RTX, Tesla, and Ampere family of products which have enough GPU memory and processing capability.
The main difference between these two types of GPUs is that the ones with virtualization usually have more GPU RAM and are offered by most cloud providers. Having more GPU RAM increases the maximum size of the image you can input to the model. In conclusion, virtualization GPUs are not significantly faster that the consumer GPUs.
You need a machine with the following hardware specifications:
| Hardware specification | Requirements |
|---|---|
| Memory |
|
| CPU |
|
| GPU |
|
| Storage |
|
For optimal performance, we recommend upgrading the GPU, as the new model is optimized for the newer Turing architecture GPUs - out of which the T4 is best suited in terms of both cost and performance. These optimizations are not available on the Pascal family of GPUs, so there will be a slight performance gap when running the new model on one.
For installations using Nvidia vGPU to work, make sure the vGPU is CUDA-enabled and the license is configured correctly.
Depending on the configuration you use (recommended GPU: Nvidia T4), you can expect the following performances (processing time is measured in seconds):
| Resolution | Inference time | Total service time |
|---|---|---|
| 1280x720 | 0.367 | 0.388 |
| 1440x900 | 0.487 | 0.515 |
| 1600x900 | 0.503 | 0.533 |
| 1920x1080 | 0.562 | 0.598 |
| 1920x1200 | 0.636 | 0.675 |
| 2560x1440 | 0.832 | 0.884 |
| 3840x2160 | 1.484 | 1.581 |