Personal Collection or Archive?: A Closer Look at What Defines a Library

archive

I was recently contacted by Dan Gravell, founder and programmer of the server-based music management software, bliss. Bliss received praise from Andrew Everard of What Hi-Fi and their official website calls it a tool “for people who care about their music collection.” Dan posed several questions about my library, and about what differentiates an average personal music collection from a true archive. He suggested that my response might prove useful as a journal entry at Innerspace Labs, so I’m sharing my response for others who might ask the same questions about their own meticulous collections.

So let’s dive right in –

Regarding the difference between run-of-the-mill “playable” music libraries and what one might call an “archive,” there are a few primary factors which could differentiate the two. The first is one of practical function and intent. If a library is for personal use for playback alone it is most likely the former, whereas a consciously organized collection of significant size and scope which is representative of a particular period or culture and which sheds contextual light on that era might serve a greater, almost scholarly purpose as an archive. Uniformity of structure, organization, navigability, and accompanying supplemental metadata enhance a library such as this to greater usefulness than mere playback. And it appears that it is precisely this focus on consistency by which Dan has endeavored to empower users like me with his bliss project. Another important factor is the long-term sustainability of an archive, which I’ll touch upon momentarily.

Next Dan asked whether my source media is exclusively physical. My collection comprises only a few thousand LPs, with a significant focus on the history of electronic sound. This spans the gamut from early notable works of musique concrète to the Moog synthesizer novelty craze, all the way through the international movement of ambient electronic music. I’ve also a predilection for archival box sets, like the Voyager Golden Record 40th Anniversary set with companion hardcover book and the special release from The John Cage Trust, as well as the previously unreleased collection of Brian Eno’s installation music issued earlier this year on vinyl with a new essay by Eno. But the bulk of my library is digital. This is both for practical and financial reasons, as digital libraries are far easier to maintain. (I don’t blog about digital nearly as often, as 450,000 media files are nowhere near as fascinating as a handsome limited edition LP!)

Dan also inquired about my workflow, which is critical to any archive. Early on in the development of my library, (around 2002-3), I began ripping LPs with the following process:

Exclusive analog recordings are captured using a Denon DP-60L rosewood TT with an Ortofon 2M Red cart, powered by a McIntosh amplifier (later replaced with a vintage Yamaha unit), and are saved as lossless FLAC via an entry level Behringer U-Control UCA202 DAC. I previously utilized a Cambridge Audio DacMagic DAC but after it failed I opted for the Behringer and it has been more than sufficient for my needs. Audio is captured using Audacity on my Linux-based DAW and basic leveling and noise reduction are performed but I minimize post-processing to maintain as much of the original audio’s integrity as possible.

Dan specifically inquired as to where the library information was stored (barcodes, etc) and asked about my policies on which metadata are included. This is fairly straightforward, as nearly all of the vinyl recordings I ripped pre-date the use of barcodes or were limited private releases with only a catalog number, which I bracket as a suffix in the release folder path.

Polybagged LPs are stored vertically and organized by primary genre, then by artist, then chronologically by date of issue. Due to the entropic property of vinyl playback, discs are played once as needed to capture the recording and subsequent playback is performed using the digital files. I employed a dozen static local DB applications over the years for my records, but eventually migrated to a Discogs DB which increases accessibility while crate digging in the wild and provides real-time market value assessment for insurance purposes.

But honestly, I almost never need to perform the rip myself, as the filesharing ecosystem has refined itself to the point where even the most exclusive titles are available through these networks in lossless archival FLAC with complete release details. There has never been a better time to be alive as an audio archivist.

Once digitized to FLAC, my assets are organized with uniform file naming conventions with record label and artist parent folders and parenthetical date of issue prefixes for easy navigation. gMusicBrowser is my ideal playback software for accessing large libraries in a Linux environment. Release date and catalog numbers have been sufficient metadata identifiers, as subsequent release details are only a click or a tap away on Discogs. Occasionally I will include a contextual write-up in the release folder where warranted, like in the case of William Basinski’s The Disintegration Loops 9LP + 5CD + DVD set as it related to the events of 9/11.

Next Dan inquired about how my archive is accessed. I employ Sindre Mehus’ Subsonic personal server application on my Linux DAW to make all of my audio and music video film content accessible from my phone, tablet, or any web-enabled device. I use both the official Subsonic app and the independently-developed Ultrasonic fork by Óscar García Amor for remote access of my library, (about eight hours daily). You can see a short video walkthrough of the features of the app that I put together here:

To return to his initial question about what differentiates a playback collection from an archive, my own library incorporates a few key factors which might lend itself to the latter:

– lossless bit-perfect FLAC wherever possible
– index documentation
– a systematic process guide for new acquisitions
– a 76pp manual highlighting special collections and large libraries of the Collection
– disk mirroring in multiple physical locations for preservation and sustainability
– fire protection for further indestructibility
– routine disk operation tests to mitigate risk of data loss
– complete discographic record label chronologies suffixed with catalog numbers
– elementary data visualizations created using Gephi and Prezi web-based tools
– the use of TrueCrypt whole disk encryption to prevent unauthorized access
– and the active use of Subsonic and Ultrasonic for enhanced accessibility

And scale is another noteworthy factor in my circumstances. Just to cite one example, I’ve collected every LP and single issued by the electronic duo Underworld that I’ve been able to get my hands on, and the digital audio branch of my Underworld collection comprises 482 albums, EPs and singles, including 2850 tracks and DJ sessions totaling well over 385 hours of non-stop music, spanning 36 years of Karl Hyde and Rick Smith’s work in all of their many incarnations. This collection is uniformly tagged, organized into a network of categorical root folders, and substructured into chronological subfolders by date of release. And the complete record label collections are a definite differentiator from the majority of casual-listening libraries.

I understand that my archive is small compared to the 12-20 TB libraries of some more seasoned users, but I feel that discretion and selectivity are virtues of my personal collection so that I can focus on only the most exquisite and remarkable recordings of my principle genre foci.

So what about your own collections? Do you employ standardized uniform file naming conventions and organizational standards? Do you supplement your library with relevant documentation to add context to your media? Does your collection offer insight into a particular era or musical culture? And do you take measures to ensure the longevity and sustainability of the work? If so… you might just have an archive.

Supplemental Note:

A good friend was kind enough to offer his thoughts about what sets an archive apart from other collections, and his remark was too good not to share. He said –

I think another major difference between the average personal collection and an archive is retention and adaptation.

A casual listener or collector wouldn’t have the retention of a true archive. The individual may build some playlists or even some advanced structure for locating and listening to music, but there is a very good chance that after some time, that particular music will get buried by the newer, or the most current thing the user is listening to. The casual listener may not want the huge or growing library, so when they feel they have moved on, the music will be removed from their collection. I cannot see someone who is keeping an archive remove anything from their collection. So retaining the entire collection and not removing anything because they are bored with it would be a difference.

I also mentioned adaptation. This is a rather basic idea but would be rather important in the grand scheme of things. Lets say you have a collection of 100 songs, all with 4 points of meta data. You realize as you begin to add more songs to your collection, a 5th point of data is needed. A casual listener may leave those 100 songs in the current state they’re in, with the 4 points of data. The archivist would need to go back, and add that 5th point to all 100 songs, and the new ones. Add another zero to those numbers and that can be a daunting, but necessary task for the archivist.

I really appreciated his input!

The Rise of the Collective Market

collective1-540x405_610_300_s_c1_center_center

Over the course of the last decade, we have seen a significant transition of power – the stranglehold of the market loosening from the hand of the corporate gatekeepers as they are largely replaced by more efficient systems built by the citizens of the internet.

These markets crowd-source the knowledge of community members who are proficient in a particular field of interest, who develop databases, forums for discussion, and flat-hierarchal markets in which to distribute goods far more effectively than by previous corporate models.

For example; Abebooks and Alibris each do a magnificent job of empowering consumers and booksellers alike, by creating an easily navigable flat structure marketplace where bookshops large and small can offer their titles to a global community without any additional overhead.  This creates a buyer’s market where millions of titles are available at impressively low prices.

abebooks-logo.gif

Discogs is another successful user-supported market.  The site’s users construct and maintain a detailed database and thriving marketplace of millions of music titles ranging from Billboard chart toppers to incredibly rare test pressings.  By adhering to a core, (and greatly facilitated) organizational structure of data submission, the site is able to crowd-source a vast and well-organized database.  The site also automates personal collection appraisals based on market history, right down to the condition of each item.  The site even offers catalog submissions via UPC scanning to make library building a snap.  And its marketplace is empowering for record sellers great and small as well as for music consumers the world over.  Like other online markets, there are significant cycles of inflation, but regulation likewise occurs naturally.

Discogs.Logo

Etsy offers a market for artisanal creative projects.  And Audiogon is a community to help educate users about pro audio gear with both a forum and a trade-and-sell market of its own.  For every need that arises, knowledgeable users in the community establish a market specializing in that service.  This is a core tenant of the cooperative nature of the internet community.

Audiogon.jpg

As with any eBusiness construct, several key advantages separate these ideal virtual markets from the antiquated corporate retail brick-and-mortar chain stores which came before them.  Firstly, their operating overhead is minimal to non-existent, whereas physical stores must constantly grapple with expenses like construction, maintenance, electricity and heat, staffing expenses, and insurance.  And the physical limitations of a building cripple a store front’s merchandise selection which is often restricted further by the distributors with which the corporation has aligned itself.

Target.Colgate Toothpaste Screenshot

By stark contrast, online markets shed all of the restrictions of physical space.  Most of these markets are user-supported so little staffing is required, and buyers can purchase any of millions of available products from other users anywhere in the world without corporate loyalty to a particular supplier.

These independent markets are far superior to their predecessors in every way, disseminating operating expenses and rendering the monopolistic behemoths obsolete and irrelevant.  And as digital media rises to overtake the physical goods market, this obsolescence will only exponentially increase.

We are witnessing the end of the gatekeeper era.  The Net has given rise to a new and better model of distribution –  marketplaces which empower buyers and sellers alike.  These markets, built upon fundamental automation structures and cooperative operation far more effectively serve the interests of the community.

As John Perry Barlow famously declared in his Declaration of Independence of Cyberspace to the governments of the world:

Cyberspace does not lie within your borders.  Do not think that you can build it, as though it were a public construction project.  You cannot.  It is an act of nature and it grows itself through our collective actions…

You have not engaged in our great and gathering conversation, nor did you create the wealth of our marketplaces…

You are trying to ward off the virus of liberty by erecting guard posts at the frontiers of Cyberspace.  These may keep out the contagion for a small time, but they will not work in a world that will soon be blanketed in bit-bearing media.

The century-long corporate dominance of our marketplaces is at its end.  Together we have built something better which works for all of us.

We have won.

From Subsonic to Ultrasonic – Do More With Your Media!

Friday evening was a night like any other, but as it happened this particular evening inspired a change to better my circumstances and proved to be most rewarding.

I was relaxing, reading a fascinating book on copyright reform, and enjoying my latest musical acquisitions via my Subsonic media server.  But as each track concluded and the next began, I repeatedly found myself irked by a 2-second mark of silence which persistently seized my attention and vanished my cozy, zen-like musical trance.

Subsonic is a brilliant and magnificent application, but gapless playback is not among its features.  And this periodic interruption was just bothersome enough to inspire me to take pause and find a better solution.  Within a few minutes’ time, I discovered that Ultrasonic – an independently developed Subsonic client, offered continuous playback as well as genre browsing and other features not available from the official Subsonic app.

After testing the application that evening I was so delighted with the result that I set myself to the task of creating a video feature to showcase Ultrasonic and hopefully empower other users like myself to do more with their media.  Google Play reports that only ~1000 users have downloaded the app, but as you’ll see from the feature below, it’s perhaps the best under-the-radar media client out there.

Check it out!

Published in: on May 7, 2016 at 7:30 am  Leave a Comment  
Tags: , , , , , , , ,

Is anyone else getting rid of their physical media altogether?

Now that I’ve purchased my first home, it seems a great time to shed some dead weight from my material possessions. My top 3000 LPs will stay – I’ve got them neatly shelved and organized in my office. I enjoy the ritual of interacting with the medium and nothing beats gatefold artwork. But everything else – cassettes, VHS, CDs, and DVDs, all seemed pointless to keep anymore.

Today I boxed up hundreds of CDs and traded them at a local Disc Exchange for 25 cents each. The cash I made was well worth the space it freed up on my bookshelves for music literature. (Most of the reference texts I enjoy I much prefer to read in a physical format than as an ebook.)

Of my ~750 CDs I kept only a handful from artists who really shaped my listening in the 90s. I kept several 20-bit remasters of classic jazz LPs and several debut singles like Reznor’s HALO 1 Down in It, Manson’s Get Your Gunn single and the Live at the Snakepit bootleg, and the 1989 Caroline Records debut single by White Zombie, Make Them Die Slowly. But other than a handful of cassette and CD promos, it really seemed time to let the rest go.

Honestly they will function more as interesting artifacts and conversation pieces rather than as a medium for audio/video playback.

I also spotted a large box of my fiance’s home-taped VHS tapes today. I offered to have her top 5 tapes converted to AVI and the rest we can dump.

Still, I confess – I’m keeping bargain bin VHS copies of cult classics including Santa Claus Conquers the Martians, YOG: Monster From Space, and the Pee-Wee Christmas Special… this is the shit I’m going to force my grandkids to watch someday.

So what about the rest of you digitally-savvy ladies and gents? Do you still hold onto physical media?

Published in: on October 10, 2015 at 9:46 pm  Comments (2)  
Tags: , , , , , , , , ,

The End of Scrobbling – A Farewell to Last.fm

Digital music has been a fascination of mine since the turn of the millennium.  Audioscrobbler came into being in 2002 while I was in college, and the thought of sharing my listening with a global network of musical peers was exhilarating.

Audioscrobbler merged with Last.fm in 2005, taking the social element of music to a whole new level.  There were forums to discuss listening trends, metadata analysis and recommendation engines… all while independent blogging exploded onto the scene in a flood of obscure music fetishism.

audioscrobbler

In the years since I admittedly lost touch with the service and dropped off the scrobbling radar to focus on personal relationships, collecting unscrobbleable LPs, and developing my career.  As the summer of 2015 came to a close my life was settling up nicely – I left Windows for Linux, I have a fiance, a fantastic career, and I’ve just purchased my first home.

With these stations of life secure, my mind returned to the world of scrobbling and the possibilities of merging big data and my own hyper-specific musical tastes.  I developed a ~500 day plan to scrobble every track from my library 24 hours a day for over a year to submit every title toward Last.fm’s recommendation engine.  Surely a library of over 110,000 tracks would produce some intriguing results!

But this evening, I logged into Last.fm and looked around to find that the site has retired all of its original functions.  The social forums are closed.  The “neighborhood” of your peers is now inaccessible.  The homepage offers only a most-popular-globally-this-week roster plastered with “Uptown Funk” and other predictable tracks.

last.fm top artists

The Wikipedia spelled out what I’d missed – CBS had acquired Last.fm for £140 million in 2009.  Wasting no time, in February of that year the service handed listener data over to the RIAA over concerns about a then-unreleased U2 album.  By 2010 the service closed the custom radio feature, (again over licensing issues) and in early 2015 they partnered with Spotify, further crippling the usability of the site.

But the nail in the coffin came in August of this year with their fully-overhauled website.  It received almost universally negative criticism from its users, who cited broken and missing features.

Given the new light of this information, I’m terminating the full-library scrobble project and saying farewell to Last.fm.  Still, I shall not mourn the loss for long.  The social function of digital music has experienced a parallel evolution in the world of private forums and closed groups on social media sites like Facebook.

Terry RIley - Persian Surgery DervishesA magnificent record I discovered thanks to a Facebook Record Community

Every morning I’m greeted with “now-spinning” rare vinyl treasures and independent music reviews which top anything you’d find from a recommendation engine.  One user from South Korea offered nearly 40 daily installments of records from his Tangerine Dream collection, each accompanied by a custom write-up on the featured release.

Private tracker communities, classic bulletin board systems, and other social structures of the web continue to serve as a brilliant resource for musical discovery.  Last.fm served us well during a pivotal time in the age of digital media, and it will be missed, but we’ll carry on.

last.fm

RIP Last.fm
2002 – 2015

Published in: on September 10, 2015 at 9:35 pm  Leave a Comment  
Tags: , , , , , , , , , , , ,

The Challenge: Best Strategies for Navigating the Waters of a Large Media Library

In recent weeks I’ve found my listening habits growing stagnant as my artist and label discographies are slowly exhausted.  The challenge for users with large media libraries is the task of finding yet-unexplored territories and developing strategies to facilitate the charting of those new waters.

One of the caveats of my otherwise-stellar media server software is that there is no way to browse by genre.  I realized this evening that queuing a chronology of albums from a given genre would be a wonderful way to explore new sounds within my library so I went to work straight away and by nightfall the project was a success.

A few initial discoveries – classics of soul jazz

serveimage (1)

serveimage 2

Using the genre text cloud feature in gmusicbrowser I constructed .m3u playlists of several intriguing but unfamiliar genres within my collection.  Each list  contained 10,000 to 17,000 of the tracks best-representative of the genre based upon RYM data and discographic libraries from the genre’s most prominent artists and composers.

I ended up splitting the Jazz list into two subsets – early jazz recordings from 1924-1958 and modern jazz recordings from 1959-1979.  This will help make the listening experience more uniform and will be an easier load on my mobile devices when spooling the lists.

With the task completed, I’m now ready to queue up thousands of hours of quality content from an array of genres I’d only explored superficially when I first acquired the recordings.  I’m looking forward to new discoveries and to the wonderful soundtrack it will provide for my days at the office!

The first batch of playlists are as follows:

  • Hot on the One – A Funk Odyssey
  • Ambient Worlds
  • Anatomy of a Murder: Film Noir Soundtracks
  • Beatless Space – Pure Drone
  • Beautiful Noise – 90s Dream Pop
  • Friday Nights – Intelligent Drum & Bass
  • 30 Years of Music from the Hearts of Space
  • Ninja Tune – The First 150 Albums
  • Psybient Dreams
  • Cinematic Soundscapes – Music for Films
  • The Chill Out Room – Downtempo Classics
  • The Imaginarium – Early Gypsy Jazz
  • The World of Jazz (1924-1958)
  • The World of Jazz (1959-1979)

Time to start listening!

Published in: on August 20, 2015 at 9:53 pm  Leave a Comment  
Tags: , , , , , , , , , ,

gmusicbrowser – An Ideal Solution for Large Library Management in a Linux Environment

The final task in my transition to a Linux environment was to customize a powerful music library manager and player to work for my needs.

The most intriguing contender was gmusicbrowser – a robust utility with impressive handling for libraries in excess of 100,000 tracks, and best of all – a fully-customizable interface.

This evening I came upon a magnificent library of gmusicbrowser interface layouts from vsido.org with an accompanying step-by-step installation guide. After about 30 minutes of perusing the 40-odd layouts bundled in the collection I came upon one which wowed me. A few minor tweaks later and I found myself with a large library manager and player with an incredibly powerful interface which permits me to fully-indulge my metadata fetishism.

Have a look – this is better than anything I had Windows-side! Highly recommended for Linux users with archival collections!

Screenshot from 2015-08-15 21:39:44

(Click/Tap to enlarge for it’s full glory)

Published in: on August 15, 2015 at 10:15 pm  Leave a Comment  
Tags: , , , , , , , , , , , ,

Leaving the Cloud for my own Private Island

I spent the last two years fully-embracing the cloud. And why not? Cloud computing offers many wonderful features. Google’s suite of apps create a seamless user experience from one personal device to another. Sites like Discogs.com empower users to access and share their record catalogs everywhere they go. Goodreads.com networks book-lovers from all around the world and democratized the used book market by facilitating the search and purchase of titles. It created a market where even the tiniest, tucked-away bookshops could compete directly with bookselling giants like Barnes & Noble.

I use Google Docs for drafts of articles I’m writing and really enjoy the flexibility of calling them up on my phone, tablet, or my media workstation throughout my day. And I’ve absolutely lived on Google Calendar for many years now.

Cloud-based archival storage services offer users data redundancy and reliable sync-and-forget-it backup systems with a 99.9% recovery rate – far more reliable that entrusting all your precious data to a single external disk.

But recently, I’ve been rethinking the cloud, particularly about the amount of control and privacy a user relinquishes when their content is no longer stored locally. iTunes was the worst atrocity to come of the cloud, as many users are starting to understand. The DRM fiasco crippled the usability of the software, and as users learned from the U2 incident, their music libraries were really at the mercy of Apple.  Spotify and streaming services are not much better, with drastically-limited media selections and, again, the content is never really yours.

iPod DRM

The entire era of cloud-computing was less about empowering the user and more of an exercise is usury. Let’s face it – storage has become incredibly inexpensive. And the popularity of lossy-compression for casual listening has only made it easier and cheaper for users to have it all. There was no longer a need to up-sell a customer base to a bigger and better device every six months, because the average smartphone suits most users just fine as an all-around media player.

For those with more discerning tastes, a simple and inexpensive home server is sufficient to grant instant-access to terabytes of lossless audio and HD video libraries from our tablets and phones anywhere with 4G service.

The industry had to invent a new way to maintain a steady influx of customer revenue. Enter the streaming service and world of online backups. These subscription-based services keep the customer paying month after month for storage and instant-access. Adobe was perhaps the most curious company to go this route, releasing the latest version of its software suite rebranded as the Creative Cloud. The customer scenario was much the same for Adobe – previous versions were everything their customers needed, so why would they need to upgrade ever again? The solution was clear – monthly subscription fees.

Adobe-survey-CC-pricingCNET Adobe CC Pricing Survey (2012)

The elephant in the room of cloud computing is the compromise of one’s privacy and security. Facebook users know all too well that every minute detail of their publicly-broadcasted lives is being sold and re-sold to advertisers banking on hyper-targeted marketing.

But you know all this – you don’t buy in to cloud archive services. You’ve implemented all the standard privacy tools and ad-blocking plugins and your web experience is fairly secure and advert-free. But what about those who don’t have the luxury of their own media server or truly unlimited data plans for their portable devices? How should they freely access their large libraries of media anywhere they go?

There is a solution. Seagate manufactures a device specifically tailored to meet the needs of this particular niche of customers and to resolve their unique problem. The Wireless Plus 2TB portable HDD (STCV2000100) is surprisingly compact and lightweight. It features an internal 10-hour battery and its own personal WiFi network. Pair it with each of your personal devices and you’ve got 2TB of content with you EVERYWHERE – on or off the grid, with no monthly fees.

Seagate Wireless Plus 1

Currently priced at ~$190, the Wireless Plus offers an incredible amount of freedom for its price point. For users like myself with our own media servers, there really isn’t an urgent need, (save perhaps for taking your entire library with you on a camping trip.) But for those tired of shelling out monthly fees for remotely hosted content – this is the device you’ve been waiting for.

Seagate Wireless Plus 2