For the folks that are impatient, the POC of DeepSeek is here KarMeng / deepseek-simple / localdev — Bitbucket

For a personal level, DeepSeek model released in just before the Chinese New Year is a biggest gift a technical person can receive. Due to friends and family, the effort to look into this model has to take the back seat, Chinese New Year time for friends and family.

The total amount of time spent to create this POC with help of both DeepSeek and ChatGPT, to co-pilot backend, frontend took a little more than 16 working hours.

The POC is built in an oracle VM, with 2 vCPU and 16GB dockerized the DeepSeek distilled model served by gunicorn running a Python flask web application.

The initial roadblock faced was interfacing the gunicorn with the frontend. Web browsers in the developer tools showing CORS error. Due to lack of experience handling modern front end, I had spent 80% of the time co-pilot with ChatGPT and DeepSeek to resolve the challenge.

Another unexpected challenge were due to DeepSeek backend container taking more than 20GB of space causing root folder to be fulled and the docker build crash. This is easy to solve. Adding a 40GB virtual drive to the VM. After the virtual drive added, change the root folder of docker to the new virtual hard drive.

The best workflow to test the DeepSeek will be starting up only the backend, use curl to test.

curl -X POST http://localhost:5000/generate -H "Content-Type: application/json" -d '{"prompt": "def factorial(n):", "max_length": 50}'

Performance wise using the curl command, its expected to be slow.

I noticed that running DeepSeek, the most important expect is the performance of hard disk IO. Would highly recommend PC or server or infrastructure that are hosting DeepSeek model requires RAID, or HADOOP or both server farm that are using RAID act as a single HADOOP farm. The question will be how to server a model via HADOOP ?

Conclusion that I have when making DeepSeek via co-piloting and handling the challenges over the 16 hours are the following skills are needed:

1. Frontend developments are needed
2. Backend developments are needed
3. System administrations are needed
4. Computer Networking are needed

If the DeepSeek are to be hosted as an external facing these are the additions

1. Information Security
2. Load testing
3. Security Information and Event Management (SIEM)

IF there are need to train own model based on the DeepSeek open sourced papers

1. All the above skill and the following expertise
2. Data Engineering
3. Data Analytics
4. Data Science

This by no means, there are no other skills or expertise needed. Hence, for now most personal user best practice are to get a powerful PC with high CPU and very high IOPS. And use available resources.

My friend has recommended this method. I am thankful for his recommendation. DeepSeek Local: How to Self-Host DeepSeek (Privacy and Control)