The Ultimate Index v3.0 – The Innerspace Labs Media Exploration Master Workbook is LIVE!

It’s been a magnificently productive day at Innerspace Labs and we’ve reached what is to date our most prestigious milestone. I published a feature last March about the evolution of my life-long list-making of sound works, cinema, and literature that I’ve been meaning to explore. These lists also served to touch upon some of the special collections in my archive.

In the previous article I described how this process began with leather pocket journals, and as the scale of my library grew I began to publish annual print editions itemizing large collections.

These efforts were radically transformed several years ago when I migrated to Google Drive. But as the years passed and spreadsheets and documents multiplied, it rapidly became apparent that I needed to consolidate all of these various lists into a single, deep searchable index otherwise countless lists would be forgotten and disappear into the digital void of my Google Drive.

Thus began the Innerspace Labs Master Workbook project this past spring. Though this venture posed several new dilemmas. As the workbook grew to nearly 200 tabs, I received this error stating that Google Workbooks are limited to 5 million or fewer cells.

And it quickly became evident that navigation of all those tabs was painfully arduous in the mobile environment, as was its loading time. Thankfully, after careful research into various potential solutions, I’ve implemented a system of scripts and formula expressions which make navigating this large workbook a snap and its interactive response time nearly instantaneous.

By combining over 200 named ranges, and incorporating a primary dynamic drop-down and a dependent secondary drop-down field, along with an “=INDIRECT(CONCATENATE” expression calling named ranges based on user input, I’m now able to hide and lock all but one master sheet and made the entire workbook navigable from that single homepage.

The home sheet offers the user a primary drop-down of LITERATURE, SOUND, or VIDEO, which in turn controls a secondary dependent drop-down to populate and auto-alphabetize a list of all related content for that category.

I’ve also employed a script which is triggered by Google Clock to rescan the entire workbook for newly-added lists and to automatically incorporate them into the search fields alphabetically and by category as the workbook continues to grow.

I understand that it may not have significant value to anyone other than myself, but it’s intended to serve as a reference document along with the over 200-pages of archive summaries I’ve drafted in a companion Google Doc. With this easy-to-reference Workbook, I can pull up a list in seconds and start exploring. My hope is that the project helps introduce me to some spectacular content and that it helps me rediscover forgotten areas of my library.

The next phase of the project is to apply uniform formatting to all lists, as these were drafted independently over the course of nearly a decade, so I apologize for the crudity of its present format. And of course, there may be errors or omissions among the lists. But you know that I’ll work tirelessly to make this project as accurate and accessible as I can.

Here is a link to a copy of the latest version. It showcases and attempts to organize ~26,000 of the most noteworthy elements of my personal library and related subjects of interest. All cells are locked for editing except the two dynamic drop downs, which is sufficient for general users to explore and interact with the document. It’s far from perfect, but it’s a labor of love that I will continue to work on and which I hope will enrich my life as it continues to expose me to some of the greatest works of the ages.

The Record Divider Project

Ever-striving to improve upon the organizational standards of The Innerspace Labs library, I finally set myself to the task of creating custom genre-labeled PVC dividers for the genre sections of my collection.

I began by assessing the key genres which would most effectively and productively be represented with tabs and compiled a list of 21 primary genres. Next, I surveyed various marketplaces for materials and determined that Rochester, NY’s classic Bags Unlimited collectors’ supply store had the best supplies available and at the lowest price compared to eBay and Amazon. (A tip – phoning in your order to BU will expedite the shipment as they do not have to transfer the materials from their web system!)

While their site is well-organized, they did not specifically provide dimension information for the tab area of their dividers nor the character width of their standard 0.5″ adhesive lettering. But with some simple importing and scaling in Gimp I was able to derive those dimensions and determine the maximum number of characters per 6″ tab, (which is approximately 12-15). I then adjusted all my genre labels, simplifying them to twelve or fewer characters.

Counting the number of each letter per sheet I dumped my list into a web-based character frequency counter and determined that I would need 9 of the shop’s sheets to complete the project. I ordered a pack of 10 to be safe. Shipping was free and they arrived in just 48 hours so I got right to work.

I had read on a scrapbooking site about the technique of using a flat acrylic ruler to aid in typesetting and in keeping the lettering centered and on a uniform baseline. Not having a typesetter’s ruler handy, and seeing that all suppliers in my area were out of stock of them, I produced one myself using a spare heavy sheet of acetate I found and  trimmed down in my workplace’s mail room, added a few 1/2″ incremental markings to aid in centering, and dove into the project.

It took just two hours from start to finish, and I photographed the results. Here are the completed set of 21 dividers just as I finished setting them.

I pre-measured my various storage systems to ensure that these standard dividers would fit and function in each space. They worked perfectly. Here they are in action. I think they add a touch of professionalism to my listening room and hope that years from now when I retire and bestow my library upon a foundation or organization of my choice that these will make the work of the recipient far easier to bear.

It was a fun accomplishment!

 

(The box sets shelf seemed sufficient on its own so I didn’t include a divider here.)

The whole project was very affordable and really enhances my library’s organization. Highly recommended for anyone looking to spruce up their listening room!

The Ultimate Index: The Innerspace Labs Media Exploration Master Workbook

February has been a whirlwind of productivity and I’m excited to share the results of my efforts. Thus far I’ve introduced five projects. First I discovered that the disk snapshot solution I’d been employing for my server would no longer work at its current scale, so I had to research and implement a new solution. Once that was a success, I set myself to the task of merging and updating two music database systems I’d created years apart on two different operating systems. That was an incredible challenge.

The next three projects were featured here at Innerspace Labs – first the Nipper RCA “His Master’s Voice” project, then the six-hour drone high-fidelity ambient experiment with Eno’s Music For Airports, followed by the Fred Deakin archive update. But it was the sixth subsequent undertaking which would consume countless late night hours as the latest project continuously exploded in scope and scale, each time introducing new challenges to test my problem-solving skills.

For as long as I’ve been breathing, I’ve been compiling and organizing lists of all manners of subjects. I thrive creating order from chaos – chronicling and curating media of the 20th-century. As a young man, I penned lists in leather pocket journals but was frustrated by the fixed and static state of the data one committed to the page. I quickly graduated to Microsoft Office and then to LibreOffice, and by 2013 began self-publishing books of collected lists and spreadsheets to document the progress of my archive.

But the true game-changer came when I adopted the Google suite of apps, most notably Google Docs, Sheets, and the Google Keep task manager. These applications introduced undo history, increased accessibility, and most importantly, shareability to my list-making efforts.

Still, the seamless convenience of Google Drive came with a caveat – scores of lists once generated were quickly forgotten, and the sheer number of them made Google Keep and Google Calendar reminders cumbersome and an ineffective method of managing them at this scale. What I came to realize was that dozens of quality sets of information were disappearing into the digital black void of a Google Drive overrun with lists.

That’s what inspired this latest project. I decided to survey my entire history of list-making, compiling databases created in a wide array of formats and constructed on multiple platforms over the years, and to merge them all into a single workbook on Google Sheets. It was an incredible challenge, as the formatting of the data varied tremendously from .M3U to .PUB to raw .TXT to .XLS to proprietary database systems built for Windows XP (OrangeCD), to web-based database systems like Discogs and Goodreads which each offered .CSV exports.

To depict folder-structure-based organizational systems, (commonly employed for artists and label discographies), I utilized tree -d list.txt for large libraries. To extract %artist% and %title% metadata from RYM toplist playlists I’d constructed, I developed a spreadsheet combining four formulas to pull nth row values and to truncate “#EXTINF:###,” expressions and file paths from .M3U lists outputting a clean list of tracks.

In October of 2017 I’d authored The Innerspace Labs Journal: A Listener’s Guide to Exploration in Google Docs as a contextual survey of my larger collections. It spans eighty-four pages and includes an active hyperlinked TOC with an X.XX indexing structure and served my needs well for the past two years, but for simple down-and-dirty lists a spreadsheet seemed like a more accessible format.

And so I constructed this latest effort – The Innerspace Labs Media Exploration Master Workbook – a cloud-based 180-tab set of spreadsheets combining all of my list data into a single, searchable, sharable index with a hyperlinked Table of Contents for easy navigation. The interface is intuitive, it loads lightning fast on even the most modest of systems and across all browsers and platforms, is mobile-friendly, and it will continue to grow as new content is introduced to my library.

The TOC is segmented into four primary themes:

1. Literature and Essays
2. Cinema and Television
3. Sound Pt 1: Music Surveys, Best-Of Lists, and Guides
4. Sound Pt 2: Artist Discographic Chronologies, Audiobooks, and Old-Time Radio Dramas

While a few of the tabs contain hyperlinks to lists from multi-page sites which do not send themselves well to text extraction, I’ve done my best to embed as much of the information as possible locally in the workbook, itself and to keep the layout consistently uniform to facilitate navigation and clarity.

Unlike the self-published books or the somewhat daunting length of the Journal, this workbook is simple and localizes the data a viewer is most interested in exploring to a single, plaintext sheet for quick and easy reference. The shareability is key to aiding curious listeners/viewers in finding quality content relevant to their interests, and it is simultaneously a tool to empower me to delve into the many areas of my own library which I’ve yet to explore.

This is a milestone for Innerspace Labs, and I will continue to refine and expand the project into the future.

Personal Collection or Archive?: A Closer Look at What Defines a Library

I was recently contacted by Dan Gravell, founder and programmer of the server-based music management software, bliss. Bliss received praise from Andrew Everard of What Hi-Fi and their official website calls it a tool “for people who care about their music collection.” Dan posed several questions about my library, and about what differentiates an average personal music collection from a true archive. He suggested that my response might prove useful as a journal entry at Innerspace Labs, so I’m sharing my response for others who might ask the same questions about their own meticulous collections.

So let’s dive right in –

Regarding the difference between run-of-the-mill “playable” music libraries and what one might call an “archive,” there are a few primary factors which could differentiate the two. The first is one of practical function and intent. If a library is for personal use for playback alone it is most likely the former, whereas a consciously organized collection of significant size and scope which is representative of a particular period or culture and which sheds contextual light on that era might serve a greater, almost scholarly purpose as an archive. Uniformity of structure, organization, navigability, and accompanying supplemental metadata enhance a library such as this to greater usefulness than mere playback. And it appears that it is precisely this focus on consistency by which Dan has endeavored to empower users like me with his bliss project. Another important factor is the long-term sustainability of an archive, which I’ll touch upon momentarily.

Next Dan asked whether my source media is exclusively physical. My collection comprises only a few thousand LPs, with a significant focus on the history of electronic sound. This spans the gamut from early notable works of musique concrète to the Moog synthesizer novelty craze, all the way through the international movement of ambient electronic music. I’ve also a predilection for archival box sets, like the Voyager Golden Record 40th Anniversary set with companion hardcover book and the special release from The John Cage Trust, as well as the previously unreleased collection of Brian Eno’s installation music issued earlier this year on vinyl with a new essay by Eno. But the bulk of my library is digital. This is both for practical and financial reasons, as digital libraries are far easier to maintain. (I don’t blog about digital nearly as often, as 450,000 media files are nowhere near as fascinating as a handsome limited edition LP!)

Dan also inquired about my workflow, which is critical to any archive. Early on in the development of my library, (around 2002-3), I began ripping LPs with the following process:

Exclusive analog recordings are captured using a Denon DP-60L rosewood TT with an Ortofon 2M Red cart, powered by a McIntosh amplifier (later replaced with a vintage Yamaha unit), and are saved as lossless FLAC via an entry level Behringer U-Control UCA202 DAC. I previously utilized a Cambridge Audio DacMagic DAC but after it failed I opted for the Behringer and it has been more than sufficient for my needs. Audio is captured using Audacity on my Linux-based DAW and basic leveling and noise reduction are performed but I minimize post-processing to maintain as much of the original audio’s integrity as possible.

Dan specifically inquired as to where the library information was stored (barcodes, etc) and asked about my policies on which metadata are included. This is fairly straightforward, as nearly all of the vinyl recordings I ripped pre-date the use of barcodes or were limited private releases with only a catalog number, which I bracket as a suffix in the release folder path.

Polybagged LPs are stored vertically and organized by primary genre, then by artist, then chronologically by date of issue. Due to the entropic property of vinyl playback, discs are played once as needed to capture the recording and subsequent playback is performed using the digital files. I employed a dozen static local DB applications over the years for my records, but eventually migrated to a Discogs DB which increases accessibility while crate digging in the wild and provides real-time market value assessment for insurance purposes.

But honestly, I almost never need to perform the rip myself, as the filesharing ecosystem has refined itself to the point where even the most exclusive titles are available through these networks in lossless archival FLAC with complete release details. There has never been a better time to be alive as an audio archivist.

Once digitized to FLAC, my assets are organized with uniform file naming conventions with record label and artist parent folders and parenthetical date of issue prefixes for easy navigation. gMusicBrowser is my ideal playback software for accessing large libraries in a Linux environment. Release date and catalog numbers have been sufficient metadata identifiers, as subsequent release details are only a click or a tap away on Discogs. Occasionally I will include a contextual write-up in the release folder where warranted, like in the case of William Basinski’s The Disintegration Loops 9LP + 5CD + DVD set as it related to the events of 9/11.

Next Dan inquired about how my archive is accessed. I employ Sindre Mehus’ Subsonic personal server application on my Linux DAW to make all of my audio and music video film content accessible from my phone, tablet, or any web-enabled device. I use both the official Subsonic app and the independently-developed Ultrasonic fork by Óscar García Amor for remote access of my library, (about eight hours daily). You can see a short video walkthrough of the features of the app that I put together here:

To return to his initial question about what differentiates a playback collection from an archive, my own library incorporates a few key factors which might lend itself to the latter:

– lossless bit-perfect FLAC wherever possible
– index documentation
– a systematic process guide for new acquisitions
– a 76pp manual highlighting special collections and large libraries of the Collection
– disk mirroring in multiple physical locations for preservation and sustainability
– fire protection for further indestructibility
– routine disk operation tests to mitigate risk of data loss
– complete discographic record label chronologies suffixed with catalog numbers
– elementary data visualizations created using Gephi and Prezi web-based tools
– the use of TrueCrypt whole disk encryption to prevent unauthorized access
– and the active use of Subsonic and Ultrasonic for enhanced accessibility

And scale is another noteworthy factor in my circumstances. Just to cite one example, I’ve collected every LP and single issued by the electronic duo Underworld that I’ve been able to get my hands on, and the digital audio branch of my Underworld collection comprises 482 albums, EPs and singles, including 2850 tracks and DJ sessions totaling well over 385 hours of non-stop music, spanning 36 years of Karl Hyde and Rick Smith’s work in all of their many incarnations. This collection is uniformly tagged, organized into a network of categorical root folders, and substructured into chronological subfolders by date of release. And the complete record label collections are a definite differentiator from the majority of casual-listening libraries.

I understand that my archive is small compared to the 12-20 TB libraries of some more seasoned users, but I feel that discretion and selectivity are virtues of my personal collection so that I can focus on only the most exquisite and remarkable recordings of my principle genre foci.

So what about your own collections? Do you employ standardized uniform file naming conventions and organizational standards? Do you supplement your library with relevant documentation to add context to your media? Does your collection offer insight into a particular era or musical culture? And do you take measures to ensure the longevity and sustainability of the work? If so… you might just have an archive.

Supplemental Note:

A good friend was kind enough to offer his thoughts about what sets an archive apart from other collections, and his remark was too good not to share. He said –

I think another major difference between the average personal collection and an archive is retention and adaptation.

A casual listener or collector wouldn’t have the retention of a true archive. The individual may build some playlists or even some advanced structure for locating and listening to music, but there is a very good chance that after some time, that particular music will get buried by the newer, or the most current thing the user is listening to. The casual listener may not want the huge or growing library, so when they feel they have moved on, the music will be removed from their collection. I cannot see someone who is keeping an archive remove anything from their collection. So retaining the entire collection and not removing anything because they are bored with it would be a difference.

I also mentioned adaptation. This is a rather basic idea but would be rather important in the grand scheme of things. Lets say you have a collection of 100 songs, all with 4 points of meta data. You realize as you begin to add more songs to your collection, a 5th point of data is needed. A casual listener may leave those 100 songs in the current state they’re in, with the 4 points of data. The archivist would need to go back, and add that 5th point to all 100 songs, and the new ones. Add another zero to those numbers and that can be a daunting, but necessary task for the archivist.

I really appreciated his input!

Fall 2015 Megapost: The Playlist Project

This summer brought many changes to The Innerspace Library.  First we started fresh with a Linux OS and finally said “farewell” to Windows.  There was a brief period of limbo as I tested various open source media management software to find the right fit for my collection.  I finally settled down with gmusicbrowser which outperformed Clementine and other major players in its handling of large libraries and in the incredible versatility and customization of its GUI.

This was the very first time since the launch of Winamp 5 (the amusing successor to  Winamp 3) that I’d explored the power of music metadata to organize my library dynamically across multiple data points.  (I’d never really saw the need during my years with MediaMonkey Gold.)

But as the summer drew to a close, I was still irked  that my Subsonic media server lacked the function of genre browsing.  I’d previously sidestepped this issue by generating mammoth genre playlists to serve as my personally-themed radio stations, each with hundreds or even thousands of the finest albums of their respective genre.

But it was this fresh start in the last few weeks that inspired my refinement of those playlists into distinct album libraries which would zero in on a specific moment of music history.  The aim was to bring a semblance of order to the hundred thousand plus tracks in my file library and to give me a set of starting points to really explore the neglected and unplayed folders of my drive.

I’m proud to declare that this evening, the project was a complete success.  I’ve created 100 all-killer-no-filler libraries showcasing each of the largest collections in my catalog.  I found that 68% (9,300 albums) of my music library fell neatly into one of these 100 categories.

The following is an index of these 100 playlists, sorted by number of albums.  This roster effectively summarizes and gives order to what is otherwise an insurmountable archive.  I’m going to enjoy exploring these playlists throughout the fall and into the winter months.

Playlists by Descending Size (in # of Albums)

Playlists with 1000+ Albums

Midnight on Mars: Ambient Worlds – 2,986 Ambient Albums

Hearts of Space: Innerspace Journey – 30 Year Complete 1,069 Broadcast Archive

Playlists with 200-999 Albums

Kelly Watch the Stars – 607 Classic Albums of the Downtempo Genre

Mentalism: Psybient Dreams – 545-Disc Archive of Psybient Electronic Music

Echowaves: Intergalactic Radio – 450 Legendary Krautrock Albums

The Shape of Jazz to Come – 387 Modern Jazz LPs (1959-1979)

Just Gimme Indie Rock!: 379 of the Greatest Indie Rock Albums (1988-2014)

Underworld: Dark & Long – A 35 Year Chronology – 339 Albums from Screen Gemz to Eno & Hyde

Old Time Radio: Dragnet (298 Broadcasts)

Salute to Birdland – 259 Classic Jazz Records (1924-1958)

FAX +49-69450464 Label Archive: The Legacy of Pete Namlook 254 Disc Catalog

We Came to Funk Ya – A 227 Album Funk Odyssey

Ninja Tune: Turn Me Loose – 204 LP Archive of Ninja Tune Records

Playlists with 100-199 Albums

Days of the Lords: 195 Album Archive of Ethereal & New Wave, Gothic Rock, Minimal Wave, Post-Punk, Jangle & Noise Pop (1976-1997)

The KLF: Abandon All Art Now – 189-disc Catalog of the Justified Ancients of MuMu

Shirt Tail Stomp: Swing & The Big Bands – 181 LP and Broadcast Archive

Tangerine Dream: Journey Through a Burning Brain – 178-Disc Chronology of TD & its Side Projects

Old Time Radio: The Adventures of Superman (171 Broadcasts)

Night Lines – 140 Album Archive of Deep House Sessions

Light Patterns – Jazz of Tomorrow – 139 Future Jazz Albums

Heaven or Las Vegas: 30 Years of Dream Pop & Ethereal Wave (134 Album Archive)

Max & Dima: Sapovnela Studio Sessions – 131 Deep House DJ Sets

Nurse with Wound: Walking Like Shadow – 127 Album Discography

Old Time Radio: X Minus One – 122 Broadcasts (1955-1973)

Deutsche Grammophon: 111 Years of DG (111-Disc Box Set)

Lemon Jelly : Going Places – 110-Disc Catalog of All Things Jelly

Miles Davis: The Complete Prestige & Columbia Recordings – 109 LPs released between 1955-2014

Daft Punk: Daftendirekt – 104-LP Chronology

Flea Market Funk – 100-Disc Archive of Funky Soul & Rare Groove

Frank Zappa: Shut Up ‘n Play Yer Guitar – 100-disc Catalog (1966-2006)

RYM’s Top 100 Downtempo & Trip Hop Albums

Playlists with 75-99 Albums

Franz Liszt: Lisztomania! – 97 LP Archive

Thelonious Monk: Straight No Chaser – 97 LP Discography

Good Looking Records: 94-Disc Archive of LTJ Bukem’s Intelligent D’n’B Label

Franklin Mint: The 100 Greatest Classical Recordings – 88 LP Catalog

Somnium – 87-Album Library of Pure Drone Music

Aphex Twin: We are the Music Makers – 86 Album Chronology of Richard D James

The Electronic Brain: Future Sounds of London & Amorphous Androgynous 86-Disc Complete Chronology

Mike Oldfield: Tricks of the Light – 86 LP Discography (1973-2010)

Jimmy Smith: Jazz Scattin’ – 85 LP Discography

Cinematic Soundscapes – 83-Disc Library of Music for Films

Old Time Radio: CBS Radio Mystery Theater – The New Adventures of Sherlock Holmes (83-Disc Box Set)

Klaus Schulze: I Sing the Body Electric – 81 LP Discography

Ludwig Van Beethoven: The Bicentennial Collection – 80 LP Complete Works

Old Time Radio: BBC Radio – The Adventures of Sherlock Holmes (79-Disc Set)

Vangelis: Poem Symphonique – 77 Album Discographic Archive

Old Time Radio: The Shadow – 75 Original Broadcasts (1937-1954)

Playlists with 50-74 Albums

Sun Ra: Cosmic Tones for Mental Therapy – 74-Disc Catalog (1957-1994)

John Cage: A Chance Operation – 73 LP Discography

2manyDJS – This is Radio Soulwax: 72 Mashup Sessions

Brian Eno: Strange Overtones – 70 LP Discography (1972-2015)

Peter Gabriel: Here Comes the Flood – 68-Disc Catalog (1977-2010)

Ornette Coleman: Change of the Century – 66 LP Discography

Prayer for the Paranoid: 66 Albums from a Decade of Shoegaze (1993-2003)

Muslimgauze: The Broken Radio of Istanbul Station – 63 Album Discography

The Piano Has Been Drinking: The Complete Recordings of Tom Waits – 63 LPs (1973-2011)

Art Blakey & the Jazz Messengers: Just Coolin’ – 62 Album Discography

Karlheinz Stockhausen: Kontakte – 61 LP Archive

Old Time Radio: The Complete Sherlock Holmes Audiobooks (Unabridged 60-Disc Set)

Philip Glass: Glasspieces – 60 LP Discography of Operas, Symphonies, Sonatas and Scores

Cornelius: Out of Phase – 60 Album Discography

Spacemind: Wonderland Syndrome – 60 Psybient DJ sets

Cafe del Mar: Step into the Sunshine – 59-Disc Archive of the Sounds of Ibiza

Throbbing Gristle: 555 Jazz Funk Greats – 56 Album Discography

DJ Food – Solid Steel – 55 LPs of Classic Jazz Breaks

Herbie Hancock: One FInger Snap – 52 Album Discography

Ash Ra Tempel & Manuel Gottsching: Deep Distance – A 40 Year, 50 LP Chronology

Houdini’s Musical Box: Early Experimental Electronic Music (1940-1976) – 50 LP Archive

Porcupine Tree: Synesthesia – 50 Album Discography (1983-2013)

Playlists with 10-24 Albums

From Murmur to Monster: 21 Key Albums of the Jangle Pop Era (1983-1994)

Old Time Radio: Orson Welles Mercury Theater 1938 (20 Program Broadcasts)

Philips Prospective 21e siecle Label 18-LP Archive (1956-1972)

RYM: Round Midnight – 18 of the Highest-Rated Cool Jazz Records

Deutsche Grammophon Avant Garde: 17-LP Complete Recordings (1967-1971)

Braindance: A 15-LP IDM Chronology of Warp Records

The Plastikman Arkives: 1993-2010 (14-Disc Box Set)

Jellyroll Radio – Ragtime, Dixieland & Bluegrass Standards (13-Disc Catalog)

Claude Debussy – 12-Disc Complete Piano and Orchestral Works

Kompakt Records: Cirrus Minor – 12 Years of Ambient Music (2001-2013)

The Last.fm Project

My server is down for maintenance for the next 16 hours.  It was a perfect opportunity to begin my next long term music project.

When Innerspace Labs first switched to the cloud, I used the web-based RacksandTags service through my OrangeCD DB to create an index of all track information from my library.  Collections on the service can be searched by artist, album, or track, but lacks support for 2nd level organization like genre clustering, playlists, and other more valuable data points.

I later switched to Discogs.com.  Discogs offers real time market value assessment of your collection, but only supports physical media.  I was also disappointed to find that user-generated category foldersare not presently shareable with other users.  
As I prepared for the downtime last night, I realized that I hadn’t given Last.fm a shot since I wiped my account clean in 2014.  That year I scrobbled 30,000 tracks, but was frustrated that there was no way to submit all my library’s data without playing every track in real time.

My goal was to explore the service’s recommendation engine, and my library data would likely produce some valuable results.

So last night, I went to work.  I quickly realized that the best approach would be to queue all 100,000+ tracks and to scrobble them in order of ascending track duration.  I organized the songs into four pools of nearly equal size.  Below is a map of my library based upon these four classes – less than five minutes, less than ten minutes, less than thirty minutes, and up to 24 hours.

As the largest batch was that of the shortest tracks, there would be the greatest (and fastest) return from scrobbling these first.

I charted the play duration of each of these groupings to see what sort of timetable I’d be looking at for project completion.

Graphing the duration of each grouping clearly demonstrates that this was in fact the best course of action.

I began scrobbling immediately for the first time in a year.  Once the project is complete I’ll share some of the resulting recommendation data Last.fm provides.  I’m looking forward to it!

Happy Labor Day weekend everyone!

Digital Music Management – Taking Full Advantage of Your Data

You don’t even want to begin to re-catalog THIS.

One of the challenges of a 77,000+ song library is the hundreds of artists a user may never hear. I decided to tackle this dilemma this week and found a solution.

The first task was to shed my database systems and Explorer navigation and find a way to manage my library from within a single application and really take command of the music metadata.  I decided that, moving forward, I would reserve my database strictly for physical media, and put Media Monkey Gold to the test of managing my digital library.

To facilitate the process, it seemed appropriate to lose some dead weight. I spent the first evening methodically rounding up my adolescent-era artists and relocating them to a folder outside the primary directory, aptly titled “Plebian Audio.”

Let’s get to work.

The process of elimination was simple.

If %GENRE% includes “rock” but does not include “progressive” or “avant-garde” then [Plebian.]

If mean album track %length% < 210sec then [Plebian.]

If demographic age < 20 then [Plebian.]

Else [Primary Audio Folder]

The resulting folder was a pool of p2p-era albums. Many had not been case-corrected. The file naming conventions were anything but uniform having sourced the tracks from multiple users, and not a single artist’s folder was a complete discographic archive. These were the millennial mix-tapes of the early days of file-sharing.

The next step was to clean-up the remaining digital files. For beginners considering this task, I recommend this article to get you started.

This was followed by an evening of fine-tuning – tailoring the %GENRE% values to be consistent throughout the library.

Trip Hop / Trip-Hop
Neo-Classical / Neoclassical / Modern Classical
etc.

It was at this juncture that I made the most valuable discovery of the project – that semicolons can be employed to apply multiple values to the %GENRE% field (or to any other field) in an ID3 tag.

I used Florian Heidenreich’s lightweight but powerful MP3tag utility to batch-replace all comma separators in my library with semicolons. The process took 36 minutes, and I confess that I did not take my eyes off that progress bar for the entire duration of the task. I just kept thinking of the incredible control that multi-genre categorization would offer. For the very first time I was able to navigate my entire catalog by genre without having to tie albums to one specific tag.

The Mp3tag interface

It was a difficult lesson to learn, but digital music inherently stores all its data within its ID3 tags, making an external music database redundant. I’d dedicated considerable time to indexing my vinyl, CD and digital music into a single database, but the reality is that it stagnates your data. The realization was reinforced by the fact that private trackers no longer have a need for massive discographic torrent archives, and have instead replaced them with decentralized manageable and crowd-sourced collages. If for example a newly remastered edition of a recording is released, it can easily be introduced to the library and is seamlessly integrated into the collage with a single click – a vast improvement to the previous method, which would require reconstructing and redistributing the entire static archive.

The same principle applies to local digital music libraries. Changes or updates to tag information, whether for a single track or for 77,000 tracks is a simple task, and is instantly reflected within your digital music application.

Perhaps my epiphany is common knowledge for the general public. I’m curious, because mine is the only library I’ve ever managed. I’d be interested in seeing statistical data with regard to what database solutions the average music consumer uses to manage their own personal libraries.

Discogs.com appears to be the most popular tool for record library management at present, with the particular benefit of real-time value assessment based upon up-to-the-minute sales data from the Discogs marketplace. And recently, Libib.com has premiered as a web-based library application to manage users’ book, movie, music and video game collections. I contacted the site’s support team, but was disappointed to learn that they are entirely UPC and ISBN-based, with no plan to implement record catalog numbers or digital music data in the foreseeable future (alienating perhaps the two fastest-growing collector demographics.)

But whatever software you use for your digital library, be it foobar2000 or MediaMonkey, the applications of audio metadata are empowering. For example, I could use this new multi-genre data to host multiple web radio stations (providing I can set up a server with sufficient dedicated bandwidth.)

Which brought me to my next discovery – the concept of autoplaylists. By entering criteria such as BPM range, genre keywords, or targeted folders, MediaMonkey’s autoplaylists will automatically populate qualifying tracks, relieving the user of the burden of manually-adding new music to existing playlists.

Unfortunately, my current personal media server does not support autoplaylists, so I am considering moving from Subsonic to a MonkeyServer system. This reinforces my realization that the MediaMonkey Gold may be a single-solution for all my music management needs.

Over the next two days I created 85 primary playlists to showcase my catalog, based on the library’s most prominent genres and for artists between 30 and 381 albums to their name. With an average of a 258 hour running time per list, these playlists totaled more than 2.5 years of continuous 24hr playback and will automatically update as new qualifying recordings are added to the library.

The immediate advantage is that these playlists will surely expose me to hundreds of artists I might otherwise never have heard.

In the evenings ahead I’ll be experimenting with MM community plug-ins to really get some analytical power out of the application.

So there you have it. With a weekend project-duration of approximately 20 hours I’ve re-evaluated my cataloging system, standardized the metadata of 103,633 files, retagged every recording, constructed 85 autoplaylists and relocated my database from a static system to a dynamic environment with an incredible number of potential applications.

Below are the first pieces of data I’ve exported from the new library.  These charts were constructed based on %GENRE% data clustered by autoplaylists with run times ranging from 10 – 8,501 hours.

Happy listening everyone.