The Task Throttling using Java Concurrent Utilities (ASync) article discussed how to throttle requests asynchronously using a ThreadPoolExecutor and BlockingQueue.
It is also possible to have the throttling done by the caller threads itself synchronously. This will make the caller thread to wait if the concurrency limit is exceeded.
Taking the same scenario as example, there are only minor differences to make the throttling synchronous. We use a Semaphore to limit the concurrency. Semaphore is a synchronization mechanism that keeps a specific number of permits. Before executing the critical section, a permit is acquired from the semaphore and released once the section is complete.
- ThrottledDashboardService initializes a Semaphore with the required concurrency limit.
- While processing the post, the ThrottledDashboardService.doPost method first acquires a permit and the permit is released after execution.
- The semaphore ensures that, at any time there are only maxPoolSize threads executing the critical section.
- The critical section is executed by the caller itself. So the transaction context (if any) is retained.
- No queuing of tasks. So no question of ever increasing task backlog.
- Straightforward to send execution result to the caller.
- No buffering. Bursts in rates are handled less gracefully. The caller is made to wait till permits are available.