(not) Hammering APIs
Limiting Concurrent Promises in TypeScript
The other day I was using request promise to make some calls to flesh out a data type. Then I decided to do that roughly 600 times....
Generally speaking it is bad practice to rail an external service with hundreds or thousands of concurrent requests unintentionally. In such a situation you might want to limit the number of open connections you are maintaining against an api. An alternative way of throttling load against an external service is by limiting calls per second - but knowing the number of in-progress calls can give a better idea what kind of damage is being inflicted at any point in time; especially if the calls are long running.
Effectively, the model is to
- wrap rp.get calls
- inside the wrapper count up the number of requests in flight
- when the wrapper is invoked, if the count is above the threshold, then queue the request
- when requests complete, decrement values and check the queue
Here's a call to GET www.bing.com with request-promise.
rp.get("https://www.bing.com/search?q=cats").then(res => console.log(res)); // should log search page
And this is how we can wrap a rp.get call in our own promise.
let wrappedGet = (url: string) =>
new Promise((resolve, reject) => {
rp.get(url).then(resolve).catch(reject);
})
Final solution, where we count up/down and delay execution
import * as rp from 'request-promise';
interface IQueuedRequestElement {
resolve: <T>(p: T | PromiseLike<T>) => void;
reject: <T>(p: T | PromiseLike<T>) => void;
uri: string;
}
class RPWrapper {
private count: number = 0;
private queue: IQueuedRequestElement[] = [];
private inc() { this.count++; }
private dec() {
this.count--;
if (this.count < this.maxConcurrent) {
let next = this.queue.shift();
this.wrappedGet(next.uri).then(next.resolve).catch(next.reject);
}
}
constructor(private maxConcurrent: number) { }
wrappedGet(uri: string) {
return new Promise((resolve, reject) => {
if (this.count >= this.maxConcurrent) {
this.queue.push({ uri: uri, resolve: resolve, reject: reject });
} else {
this.inc();
rp.get(uri).then(resolve).catch(reject).finally(() => this.dec());
}
});
}
}
Software Engineer