incremental lookup refactor #18
Description
To improve performance, allow incremental updating (only update changed entries), etc a better pipeline is needed.
- To improve performance by reducing unnecessary system fetches we can expand the system developed for recovering abstracts (see Add abstracts from other sources #17) to call each services only when there's missing data from the previous services - use a configuration object to hold paths and associated functions
- To enable incremental changes, create an output file that holds the data recovered from the look up services as well as other metadata to compare and control (e.g. date of last update, etc)