Podjeli: A dumb new way to share files online

    A great many file sharing solutions have come and gone over the years. While they may differ in technology or branding, the reasons for their downfall are generally consistent. Hosting user content is challenging for three primary reasons:

    The graveyard of file sharing.
    The graveyard of file sharing.

    1. User-uploaded content and the long tail of popularity

    My guess is the median number of views for a given image uploaded to the average image host is less than two, and one of those is the original uploader. This low engagement makes storage expensive; for every image that goes viral and produces a net return (say, via advertising), thousands are stored and never retrieved.

    2. Users upload content that they shouldn't

    User content can be a ticking timebomb of legal liability. Identifying the misuse of copyrights and trademarks requires human moderators who are experts in illegal content, if not a full legal team. Responding to DMCA takedown requests can be a full-time job.

    3. Peer-to-peer solutions magnify the long tail problem

    While peer-to-peer mostly solves the illegal content liability problem, the long tail problem is compounded by peer-to-peer solutions. If nobody is interested in the content, nobody bothers to download it and serve it to other peers.

    A humble attempt at a solution

    I’ve thought about these flaws for some time, and finally decided to test the waters with a new solution. Podjeli is a file-sharing service that isn't peer-to-peer and doesn't host any files. How can you share files that haven't been stored? When someone shares a file with Podjeli, the file is compressed, chunked into small bits, and stored as query strings in a series of URLs that are then prepared as markdown, HTML, or simply made ready to be copied and pasted.

    Tech Stack

    Podjeli is a JavaScript application, and all the work, from URL generation to recombination, happens in the browser. It uses lz-string for compression and Bootstrap as the UI framework. File parts are sent as query strings appended to the URLs that are generated. Cloudflare ignores these query strings—they simply serve the same page over and over for each part, and the JavaScript does the work of loading the file parts into local storage until they can be reassembled later. Due to the limitations of local storage, only files that are 5 megabytes or smaller can be reassembled, but a file that large would result in tens of thousands of URLs, so that is an unlikely scenario.

    Opening new doors

    Store files anywhere you can store URLs

    Podjeli opens up some brand-new doors. For example, you can store one or more URLs anywhere, and you can also store an arbitrary file. Social media sites, forums, and comment sections are an obvious choice, but you should also consider things like firewall access logs or your browser’s history.

    Similarly, the browser favorites menu offers some amusing possibilities for InfoSec red teams attempting to exfiltrate information. By favoriting a list of Podjeli URLs and allowing them to sync via Chrome favorites, those files could be copied offsite relatively undetected.

    URL Shorteners + Podjeli = Smaller Files

    The consequences of Podjeli used in conjunction with URL shorteners are interesting. For example, when I took Lorem Ipsum and put it in a plain text file, it weighed in at 445 bytes. I ran that through Podjeli and created two URLs, shortened with TinyURL. Then I took those two shortened URLs and placed them in a file. The new file was 58 bytes for a savings of 86.96%. That is in line with the best English text compression available.

    The difference is even more significant with larger files. I took the same Lorem Ipsum and copied it 25 times in a text file, uploaded it to Podjeli, and created 27 URLs. From there, I ran each URL through TinyURL and saved those to a file. In this test, I went from 10900 bytes to 808. That brings the compression ratio up to 92.59%. I knew I could do a little better—I took the file containing the TinyURL URLs and ran that through Podjeli, chose the “save as text file” option, and enabled compression. The resulting zip file is 758 bytes for an effectively lossless compression ratio of 93.05%.

    A thought experiment

    Podjeli hosts no user files, but in a philosophical sense, it hosts all possible files. According to some interpretations of Multiverse theory, versions of myself have created Podjeli a infinite number of times. In theory, many of those implantations would be compatible with one another, so that a URL generated in another universe would be able to be decoded in our current one.

    Podjeli across the multiverse.

    It has also been postulated by David Deutsch that quantum computation is:

    “Nothing less than a distinctly new way of harnessing nature, it will be the first technology that allows useful tasks to be performed in collaboration between parallel universes, and then sharing the results.”

    Assuming Deutsch is correct, if someone in an alternate universe creates a file containing the cure for cancer or zero-point energy and we had access to a planet-sized supercomputer in the vein of Doug Adams we could brute force Podjeli URLs until we discover these files. Of course, this is nothing new, however I find the concept of sharing files over some sort of multiverse internet intriguing.

    Final thoughts

    Overall, Podjeli was a project that I wanted to create in order to scratch an itch of mine. I used Bootstrap and jQuery because I am familiar with those tools and they have excellent backwards compatibility. However, going forward, I will probably use something like React and Typescript. In a way, this was my goodbye to the web development tools I cut my teeth on. Bootstrap, it’s been real—see you around.

    Check out Podjeli on Github.

    Will Cipriano

    Read more posts by this author.