We have a service that allows people to submit long running jobs to us, and we process them.
We want to process these as fast as possible (obviously).
We allow our users to submit a job, and query for job status.
Jobs can be paused (at some penalty) and their progress can be saved before swapping to another job.
How do we make an efficient and fair job queue?
What happens if some jobs take longer than others? (i.e. some are simple calculations, some are I/O bound).
What happens if we want to prioritize certain tasks instead of others (they are more mission critical than others?)
What happens if we don’t know how long a job will take before accepting it?
Can we add concurrency to good effect here? What are the pitfalls with concurrent jobs running?