IDEA: Allow users to process multiple files/links at the same time

[dogzilla]

Is there really no way to batch upload files? Is open-notebook designed to only work with a handful of files? I have hundreds. Did I pick the wrong tool?

[lfnovo]

Hi Dogzilla.. we have just release an API which was the requirement to do multi-upload. This is coming up in the next release. If you are not comfortable using the API, it should be ready to use pretty soon.

[dogzilla]

Excellent news. I actually coded my own little tool to handle batch uploads via the API. It worked well, but now it looks like the notebook is too big (300+ sources) to manage via the UI, although ask & search still works fine, if a bit slow. I may try to use the query API to see if it's just the UI that's slow or the size of the notebook, but it definitely works as long as you don't mind waiting a minute or two for answers. Note that search is blazing fast using either text or vector, so it may be the combination of local llm and ui speed (the UI is slow to respond everywhere) that's slowing the ask responses.

This tool has a lot of potential - great work.

[lfnovo]

That's why I am killing this UI for a React-based one. Will be here very soon, 80% done.

[dogzilla]

I look forward to it. This is the only local tool I’ve gotten any usable results from, and the quality of summaries is excellent. I can use this directly and looking at integrating it as a callable tool in my frontend(s)

[lfnovo]

Nice man.. thanks for today's motivation rush :)

[deneb2]

Excellent news. I actually coded my own little tool to handle batch uploads via the API. It worked well, but now it looks like the notebook is too big (300+ sources) to manage via the UI, although ask & search still works fine, if a bit slow. I may try to use the query API to see if it's just the UI that's slow or the size of the notebook, but it definitely works as long as you don't mind waiting a minute or two for answers. Note that search is blazing fast using either text or vector, so it may be the combination of local llm and ui speed (the UI is slow to respond everywhere) that's slowing the ask responses.

This tool has a lot of potential - great work.

I manually added around 40 sources and the UI become quite slow. I don't have a local llm, so it is definitely the UI

[lfnovo]

Just pushing some updates here. The new UI is live today. It still does not have the ability to process multiple sources at the same time, but the source processing routine is now a background service. It should be really fast for you now and should not clog the UI. You can use the API if you want to process multiple of them at the same time. I'll eventually get to multiple sources. But, right now, the priority is to make the current new release very stable and easier to use as it is.