Converting a javascript loop to be asynchronous and functional

Question

I'm trying to make the following code more functional and asynchronous, but not having a lot of luck!

So the api calls can only take 5 id's at a time, which is why I created splitListOfIds and am looping through it.The second api call is dependent on the first api call

The code below works, but is slow and I get ESLINT errors with this code though and want to make it more functional and asynchronous. I've been trying to use Promises but not sure how to do it. Ideally want to get rid of the for loop.

const splitListOfIds = [
    [1,2,3,4,5],
    [6,7,8,9,10]
]

const allPolicies = []

for (let i = 0; i < splitListOfIds.length; i++) {
    const people = await peopleApi.getPeople(splitListOfIds[i])
    const policyIds = people.map(p => p.policyId)
    const policies = await policyApi.getPolicyDetails(policyIds)
    allPolicies.push(policies)
}

If you're using await you're already using Promises. Any specific reason why you want to get rid of the for loop? — Jacob
– Jacob, Commented Mar 18, 2020 at 15:03

Stephane Janicaud · Accepted Answer · 2020-03-18 16:13:10Z

I would use RxJs and Observable to do this in an elegant way, for example

// Emit people ids as a sequence
const source = Rx.Observable.from([1,2,3,4,5,6,7,8,9]);
const subscribe = source
.bufferCount(5) // Buffer people ids to ensure the size won't exceed 5 for api calls
.flatMap(peopleIds => peopleApi.getPeople(peopleIds))
.map(people => people.map(user => user.policyId))
.flatMap(policyIds => policyApi.getPolicyDetails(policyIds))
.subscribe(policies => console.log(policies));

Here is a snippet mocking your api

const mockPeople = [
  { "id": 1, "name": "Jack", "policyId": "p-01" },
  { "id": 2, "name": "Isobel", "policyId": "p-02" },
  { "id": 3, "name": "Steve", "policyId": "p-03" },
  { "id": 4, "name": "James", "policyId": "p-04" },
  { "id": 5, "name": "Marty", "policyId": "p-05" },
  { "id": 6, "name": "Janis", "policyId": "p-06" },
  { "id": 7, "name": "Annabel", "policyId": "p-07" },
  { "id": 8, "name": "Flora", "policyId": "p-08" },
  { "id": 9, "name": "Richard", "policyId": "p-09" },
]

const mockPolicies = [
  { "id": "p-01", "details": "Details for Jack's policy" },
  { "id": "p-02", "details": "Details for Isobel's policy" },
  { "id": "p-03", "details": "Details for Steve's policy" },
  { "id": "p-04", "details": "Details for James's policy" },
  { "id": "p-05", "details": "Details for Marty's policy" },
  { "id": "p-06", "details": "Details for Janis's policy" },
  { "id": "p-07", "details": "Details for Annabel's policy" },
  { "id": "p-08", "details": "Details for Flora's policy" },
  { "id": "p-09", "details": "Details for Richard's policy" }  
]

let mockGetPeople = async (peopleIds) => {
	let filteredPeople = mockPeople.filter(user => {
  	return peopleIds.indexOf(user.id) !== -1;
  });
	return Promise.resolve(filteredPeople);
}
let mockGetPolicies = async (policyIds) => {
	let filteredPolicies = mockPolicies.filter(policy => {
  	return policyIds.indexOf(policy.id) !== -1;
  });
	return Promise.resolve(filteredPolicies);
}


const source = Rx.Observable.from([1,2,3,4,5,6,7,8,9]);
const subscribe = source
.bufferCount(5)
.flatMap(peopleIds => mockGetPeople(peopleIds)) // mock of peopleApi.getPeople
.map(people => people.map(user => user.policyId))
.flatMap(policyIds => mockGetPolicies(policyIds)) // mock of policyApi.getPolicyDetails
.subscribe(policies => console.log(policies));

<script src="https://npmcdn.com/@reactivex/[email protected]/dist/global/Rx.umd.js"></script>

Jacob · Accepted Answer · 2020-03-18 15:37:45Z

2

If this API allows you to do requests in parallel, you could take this kind of approach:

const allPolicies = await Promise.all(splitListOfIds.map(async peopleIds => {
  const people = await peopleApi.getPeople(peopleIds)
  const policyIds = people.map(p => p.policyId)
  const policies = await policyApi.getPolicyDetails(policyIds)
  return policies;
}));

Depending on what the APIs are doing in the background, this can get you in trouble if you've got too many things in parallel. If that's the case, then you need to implement some kind of max parallelism mechanism, something like this (untested, though there are probably libraries for this already):

async function processInParallel(maxParallelism, data, taskFn) {
  const results = [];
  const inFlight = new Set();  

  for (let i = 0; i < data.length; i++) {
    while (inFlight.size >= maxParallelism) {
      // Wait for at least one to complete
      await Promise.race([...inFlight]);
    } 

    const task = taskFn(data[i]).then(result => {
      results.push(result);
      inFlight.delete(task);
    });
    inFlight.add(task);
  }

  await Promise.all([...inFlight]);

  return results;
}

// Usage
const allPolicies = await processInParallel(10, splitListOfIds, async peopleIds => {
  const people = await peopleApi.getPeople(peopleIds)
  const policyIds = people.map(p => p.policyId)
  const policies = await policyApi.getPolicyDetails(policyIds)
  return policies;
}));

edited Mar 18, 2020 at 15:37

answered Mar 18, 2020 at 15:07

Jacob

79k24 gold badges159 silver badges245 bronze badges

1 Comment

Storms786 Over a year ago

that looks promising! let me try it and will accept your answer if it does the trick

Gwinn · Accepted Answer · 2020-03-18 15:25:58Z

1

You have to map each async call to a Promise then await all of them together, like so:

async function callAsync() {

    const asyncCalls = splitListOfIds.map(async function(pieceOfListOfIds) {

        const people = await peopleApi.getPeople(pieceOfListOfIds);
        const policyIds = people.map(p => p.policyId);
        const policies = await policyApi.getPolicyDetails(policyIds);

        allPolicies.push(policies);

    });

    await Promise.all(asyncCalls);
}

Basically every async function returns a Promise that you will collect in the asyncCalls array and then with Promise.all you wait for every Promise to settle.
If you just use forEach your program will continue execution without awaiting the various promises.

EDIT:

Note: this code will execute all the API calls in parallel

edited Mar 18, 2020 at 15:25

answered Mar 18, 2020 at 15:10

Gwinn

1,4081 gold badge11 silver badges19 bronze badges

2 Comments

Storms786 Over a year ago

Correct me if I'm wrong but there's no functional difference between yours and Jacobs right?

Gwinn Over a year ago

Yes, Jacob posted while I was writing. If this solution sutis your needs, please accept Jacob's answer, as he arrived first.

Collectives™ on Stack Overflow

Converting a javascript loop to be asynchronous and functional

3 Answers 3

Comments

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related