Skip to content

[facebook 1] feat: import wall posts from Meta export#405

Merged
micahflee merged 69 commits intomainfrom
facebook-export-import
Mar 13, 2025
Merged

[facebook 1] feat: import wall posts from Meta export#405
micahflee merged 69 commits intomainfrom
facebook-export-import

Conversation

@redshiftzero
Copy link
Contributor

@redshiftzero redshiftzero commented Feb 19, 2025

Closes #398
Closes #407
Closes #409
Closes #399
Closes #410

TODO:

  • impl importFacebookArchive
  • Filter posts to import by "updated her status" in posts data file (without using text that will be in different languages)
  • File followup issues for other data types that can be imported from the Meta export file (one for "shared a post" in the posts file which is effectively like a retweet, one for "shared a group" which is sharing a group or group post), add media support, add URL support (currently commented out)
  • Only support JSON export

Screenshots

Add new import option

Screenshot 2025-02-20 at 8 09 20 PM

Explain how to download archive

Screenshot 2025-02-19 at 3 42 10 PM

Archive import

Screenshot 2025-02-19 at 6 34 16 PM

Import finished

Screenshot 2025-02-20 at 10 29 41 AM

Now you can select * from post to view the posts

@redshiftzero redshiftzero changed the title wip: import facebook data from Meta export feat: import facebook data from Meta export Feb 20, 2025
@redshiftzero redshiftzero force-pushed the facebook-export-import branch from 294e757 to 3ff7af5 Compare February 20, 2025 16:24
@redshiftzero redshiftzero changed the title feat: import facebook data from Meta export feat: import facebook Wall posts from Meta export Feb 20, 2025
@redshiftzero redshiftzero changed the title feat: import facebook Wall posts from Meta export feat: import facebook wall posts from Meta export Feb 20, 2025
@redshiftzero redshiftzero marked this pull request as ready for review February 20, 2025 16:27
@redshiftzero redshiftzero force-pushed the facebook-export-import branch from 0d473d0 to 3506cda Compare February 22, 2025 13:55
Copy link
Member

@micahflee micahflee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a merge conflict with main right now that I haven't looked into, and also I noticed the packages changed enough that I had to npm install again, so might be good to merge main into this branch.

On FacebookWizardImportPage.vue, the instructions for downloading your archive could be improved/ I'm following them now. In my test account I actually don't need to prove my identity. Here's what I see when I download my Facebook archive:

Screenshot 2025-02-28 at 12 48 19 PM

I click "Download or transfer information" (you have a typo in the instruction, it should be "or" instead of "and").

Screenshot 2025-02-28 at 12 48 30 PM

There should be a step in the instructions for which to choose. I'm assuming "Available information".

Screenshot 2025-02-28 at 12 49 22 PM

There should be another step in the instructions to click "Download to device".

Screenshot 2025-02-28 at 12 50 09 PM

We should tell the user what to click her:

  • Under "Date range" choose "All time"
  • Notify should be an email you can check
  • Format should be JSON (I assume? probably much easier to work with than HTML)
  • Media quality: High

Then click "Create files".

Screenshot 2025-02-28 at 12 52 04 PM

The current instructions say "After requesting your archive, you'll need to wait at least a day, and maybe longer. Sorry!" -- that's true for X, but I'm not sure how long you have to wait for Facebook.

Anyway I'm currently waiting for my Facebook archive, but this is what I could test so far.

@redshiftzero redshiftzero changed the title feat: import facebook wall posts from Meta export [wip] feat: import facebook wall posts from Meta export Mar 1, 2025
@redshiftzero
Copy link
Contributor Author

I see once the archive has been imported, we either have "Continue to Delete options" (which I guess is not implemented) or "Continue to Archive options". Continue to Archive options just takes us to a blank page, whereas I feel we can just redirect to the initial page for importing/building data?

Yep neither of these pages is implemented yet

@redshiftzero
Copy link
Contributor Author

Each entry in the JSON Array in the posts JSON file has a:

  • timestamp
  • attachments array
  • data array
    • may contain an entry with update_timestamp
    • may contain an entry with post text
  • title

Shares of groups have:

  • The group name in the first entry of attachments.data.0.name
  • The title will be "NAME shared a group"

Reposts have:

  • The first entry of attachments.data.0.external_context.url is ""
  • The title will be "NAME shared a post"
  • There may not be any text, but if there is, the first entry in the data array will be a the post text

A regular post:

  • has post text in the data array
  • The title will be "NAME updated her status"

An image post or video post:

  • has a media entry in attachments.data
    • It includes exif data including upload IP
  • has the post text in the data array
  • The title will say "NAME added a new photo" or "NAME added a new video"

A life event:

  • will have in the attachments array an entry with key data that has another array with the life_event key containing a date and title. Right now we're just pulling out the title and using it as the post text.

@redshiftzero redshiftzero requested a review from micahflee March 11, 2025 18:36
@redshiftzero
Copy link
Contributor Author

the link previews (stored in the export file) are now loaded into the database (#410) and rendered on the static site as part of this PR

Copy link
Member

@micahflee micahflee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested importing an archive and it works great. I realized that I need to actually wait for yet another archive because my current one doesn't include certain types of posts (including images).

What I've tested and works great:

  • Post text
  • Link previews

I'm happy to merge this as-is, but while I'm testing I just want to record everything I notice that should be handled in future PRs:

  • The Facebook view sidebar should have a nav sort of like the X view sidebar does, so you can easily jump between the different views. (Of course, those different views don't exist yet. Just right now after importing an archive, there's no easy way to get back to the database view.)
  • The FB static archive site does not embed media yet. (It successfully downloaded a video I posted to the media folder, but it's video isn't included in assets/archive.json and it's not embedded in the archive site.)
  • Emote data (likes, etc.) is not included, but I guess maybe it's just not included in the archive?

What I haven't tested (because I need to wait for a new archive):

  • import re-shared posts
  • import posts with images, including multiple images

@micahflee micahflee merged commit 21a0108 into main Mar 13, 2025
1 check passed
@micahflee micahflee deleted the facebook-export-import branch March 13, 2025 21:13
@redshiftzero
Copy link
Contributor Author

redshiftzero commented Mar 14, 2025

Regarding emote data, we don't have that for the posts on the wall the number of likes/emotes etc.
For the user's likes and reactions, we do have that content stored in: your_facebook_activity/comments_and_reactions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants