I recently came across a torrent that seems to be an archive of Reddit. It got me thinking if it would be possible to make it locally browsable. However, I also considered the possibility that someone might have already addressed this by creating a public Lemmy instance, enabling the content to be accessible from any federated instance.

  • qprimed@lemmy.ml
    link
    fedilink
    English
    arrow-up
    10
    ·
    1 year ago

    I can actually see some merit to a lemmy API accessible reddit corpus. it would be interesting to reference old reddit info in a lemmy compatible way with zero reference to reddit itself.

    doing so for the entire corpus properly (link fixups, etc) would be… challenging, but doable.

  • Andy@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    1 year ago

    Letting the LLMs source from their for free, completely invalidating the proposed licensing model at Reddit.

    • SwingingKoala@discuss.tchncs.de
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 year ago

      They only copy posts afaict. I subscribed to some of their communities but unsubscribed rather quickly as the post feed without comments usually has little value.

  • vamp07@lemm.ee
    link
    fedilink
    English
    arrow-up
    4
    ·
    1 year ago

    I’m at a loss as to why anybody wouldnwant this in the first place.

  • RagingNerdoholic@lemmy.ca
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    1
    ·
    edit-2
    1 year ago

    Am I really going to buy a 2TB drive to hold all of reddit…

    Actually, I’m pretty surprised that it’s only 2TB.

    Edit: and it looks like it’s only captured data up until about six months ago.

    Edit 2: turns out I have an available 2TB drive. Now I’m tempted. Should I bother with a VPN for this or nah?

    • InternetPirate@lemmy.fmhy.mlOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      It would be helpful if there were an instance that migrated all of this to Lemmy so that we could access it from any other instance, instead of having to download it for local browsing.