The Ultimate Index v3.0 – The Innerspace Labs Media Exploration Master Workbook is LIVE!

It’s been a magnificently productive day at Innerspace Labs and we’ve reached what is to date our most prestigious milestone. I published a feature last March about the evolution of my life-long list-making of sound works, cinema, and literature that I’ve been meaning to explore. These lists also served to touch upon some of the special collections in my archive.

In the previous article I described how this process began with leather pocket journals, and as the scale of my library grew I began to publish annual print editions itemizing large collections.

These efforts were radically transformed several years ago when I migrated to Google Drive. But as the years passed and spreadsheets and documents multiplied, it rapidly became apparent that I needed to consolidate all of these various lists into a single, deep searchable index otherwise countless lists would be forgotten and disappear into the digital void of my Google Drive.

Thus began the Innerspace Labs Master Workbook project this past spring. Though this venture posed several new dilemmas. As the workbook grew to nearly 200 tabs, I received this error stating that Google Workbooks are limited to 5 million or fewer cells.

And it quickly became evident that navigation of all those tabs was painfully arduous in the mobile environment, as was its loading time. Thankfully, after careful research into various potential solutions, I’ve implemented a system of scripts and formula expressions which make navigating this large workbook a snap and its interactive response time nearly instantaneous.

By combining over 200 named ranges, and incorporating a primary dynamic drop-down and a dependent secondary drop-down field, along with an “=INDIRECT(CONCATENATE” expression calling named ranges based on user input, I’m now able to hide and lock all but one master sheet and made the entire workbook navigable from that single homepage.

The home sheet offers the user a primary drop-down of LITERATURE, SOUND, or VIDEO, which in turn controls a secondary dependent drop-down to populate and auto-alphabetize a list of all related content for that category.

I’ve also employed a script which is triggered by Google Clock to rescan the entire workbook for newly-added lists and to automatically incorporate them into the search fields alphabetically and by category as the workbook continues to grow.

I understand that it may not have significant value to anyone other than myself, but it’s intended to serve as a reference document along with the over 200-pages of archive summaries I’ve drafted in a companion Google Doc. With this easy-to-reference Workbook, I can pull up a list in seconds and start exploring. My hope is that the project helps introduce me to some spectacular content and that it helps me rediscover forgotten areas of my library.

The next phase of the project is to apply uniform formatting to all lists, as these were drafted independently over the course of nearly a decade, so I apologize for the crudity of its present format. And of course, there may be errors or omissions among the lists. But you know that I’ll work tirelessly to make this project as accurate and accessible as I can.

Here is a link to a copy of the latest version. It showcases and attempts to organize ~26,000 of the most noteworthy elements of my personal library and related subjects of interest. All cells are locked for editing except the two dynamic drop downs, which is sufficient for general users to explore and interact with the document. It’s far from perfect, but it’s a labor of love that I will continue to work on and which I hope will enrich my life as it continues to expose me to some of the greatest works of the ages.

The Ultimate Index: The Innerspace Labs Media Exploration Master Workbook

February has been a whirlwind of productivity and I’m excited to share the results of my efforts. Thus far I’ve introduced five projects. First I discovered that the disk snapshot solution I’d been employing for my server would no longer work at its current scale, so I had to research and implement a new solution. Once that was a success, I set myself to the task of merging and updating two music database systems I’d created years apart on two different operating systems. That was an incredible challenge.

The next three projects were featured here at Innerspace Labs – first the Nipper RCA “His Master’s Voice” project, then the six-hour drone high-fidelity ambient experiment with Eno’s Music For Airports, followed by the Fred Deakin archive update. But it was the sixth subsequent undertaking which would consume countless late night hours as the latest project continuously exploded in scope and scale, each time introducing new challenges to test my problem-solving skills.

For as long as I’ve been breathing, I’ve been compiling and organizing lists of all manners of subjects. I thrive creating order from chaos – chronicling and curating media of the 20th-century. As a young man, I penned lists in leather pocket journals but was frustrated by the fixed and static state of the data one committed to the page. I quickly graduated to Microsoft Office and then to LibreOffice, and by 2013 began self-publishing books of collected lists and spreadsheets to document the progress of my archive.

But the true game-changer came when I adopted the Google suite of apps, most notably Google Docs, Sheets, and the Google Keep task manager. These applications introduced undo history, increased accessibility, and most importantly, shareability to my list-making efforts.

Still, the seamless convenience of Google Drive came with a caveat – scores of lists once generated were quickly forgotten, and the sheer number of them made Google Keep and Google Calendar reminders cumbersome and an ineffective method of managing them at this scale. What I came to realize was that dozens of quality sets of information were disappearing into the digital black void of a Google Drive overrun with lists.

That’s what inspired this latest project. I decided to survey my entire history of list-making, compiling databases created in a wide array of formats and constructed on multiple platforms over the years, and to merge them all into a single workbook on Google Sheets. It was an incredible challenge, as the formatting of the data varied tremendously from .M3U to .PUB to raw .TXT to .XLS to proprietary database systems built for Windows XP (OrangeCD), to web-based database systems like Discogs and Goodreads which each offered .CSV exports.

To depict folder-structure-based organizational systems, (commonly employed for artists and label discographies), I utilized tree -d list.txt for large libraries. To extract %artist% and %title% metadata from RYM toplist playlists I’d constructed, I developed a spreadsheet combining four formulas to pull nth row values and to truncate “#EXTINF:###,” expressions and file paths from .M3U lists outputting a clean list of tracks.

In October of 2017 I’d authored The Innerspace Labs Journal: A Listener’s Guide to Exploration in Google Docs as a contextual survey of my larger collections. It spans eighty-four pages and includes an active hyperlinked TOC with an X.XX indexing structure and served my needs well for the past two years, but for simple down-and-dirty lists a spreadsheet seemed like a more accessible format.

And so I constructed this latest effort – The Innerspace Labs Media Exploration Master Workbook – a cloud-based 180-tab set of spreadsheets combining all of my list data into a single, searchable, sharable index with a hyperlinked Table of Contents for easy navigation. The interface is intuitive, it loads lightning fast on even the most modest of systems and across all browsers and platforms, is mobile-friendly, and it will continue to grow as new content is introduced to my library.

The TOC is segmented into four primary themes:

1. Literature and Essays
2. Cinema and Television
3. Sound Pt 1: Music Surveys, Best-Of Lists, and Guides
4. Sound Pt 2: Artist Discographic Chronologies, Audiobooks, and Old-Time Radio Dramas

While a few of the tabs contain hyperlinks to lists from multi-page sites which do not send themselves well to text extraction, I’ve done my best to embed as much of the information as possible locally in the workbook, itself and to keep the layout consistently uniform to facilitate navigation and clarity.

Unlike the self-published books or the somewhat daunting length of the Journal, this workbook is simple and localizes the data a viewer is most interested in exploring to a single, plaintext sheet for quick and easy reference. The shareability is key to aiding curious listeners/viewers in finding quality content relevant to their interests, and it is simultaneously a tool to empower me to delve into the many areas of my own library which I’ve yet to explore.

This is a milestone for Innerspace Labs, and I will continue to refine and expand the project into the future.

The Rise of the Collective Market

Over the course of the last decade, we have seen a significant transition of power – the stranglehold of the market loosening from the hand of the corporate gatekeepers as they are largely replaced by more efficient systems built by the citizens of the internet.

These markets crowd-source the knowledge of community members who are proficient in a particular field of interest, who develop databases, forums for discussion, and flat-hierarchal markets in which to distribute goods far more effectively than by previous corporate models.

For example; Abebooks and Alibris each do a magnificent job of empowering consumers and booksellers alike, by creating an easily navigable flat structure marketplace where bookshops large and small can offer their titles to a global community without any additional overhead.  This creates a buyer’s market where millions of titles are available at impressively low prices.

Discogs is another successful user-supported market.  The site’s users construct and maintain a detailed database and thriving marketplace of millions of music titles ranging from Billboard chart toppers to incredibly rare test pressings.  By adhering to a core, (and greatly facilitated) organizational structure of data submission, the site is able to crowd-source a vast and well-organized database.  The site also automates personal collection appraisals based on market history, right down to the condition of each item.  The site even offers catalog submissions via UPC scanning to make library building a snap.  And its marketplace is empowering for record sellers great and small as well as for music consumers the world over.  Like other online markets, there are significant cycles of inflation, but regulation likewise occurs naturally.

Etsy offers a market for artisanal creative projects.  And Audiogon is a community to help educate users about pro audio gear with both a forum and a trade-and-sell market of its own.  For every need that arises, knowledgeable users in the community establish a market specializing in that service.  This is a core tenant of the cooperative nature of the internet community.

As with any eBusiness construct, several key advantages separate these ideal virtual markets from the antiquated corporate retail brick-and-mortar chain stores which came before them.  Firstly, their operating overhead is minimal to non-existent, whereas physical stores must constantly grapple with expenses like construction, maintenance, electricity and heat, staffing expenses, and insurance.  And the physical limitations of a building cripple a store front’s merchandise selection which is often restricted further by the distributors with which the corporation has aligned itself.

By stark contrast, online markets shed all of the restrictions of physical space.  Most of these markets are user-supported so little staffing is required, and buyers can purchase any of millions of available products from other users anywhere in the world without corporate loyalty to a particular supplier.

These independent markets are far superior to their predecessors in every way, disseminating operating expenses and rendering the monopolistic behemoths obsolete and irrelevant.  And as digital media rises to overtake the physical goods market, this obsolescence will only exponentially increase.

We are witnessing the end of the gatekeeper era.  The Net has given rise to a new and better model of distribution –  marketplaces which empower buyers and sellers alike.  These markets, built upon fundamental automation structures and cooperative operation far more effectively serve the interests of the community.

As John Perry Barlow famously declared in his Declaration of Independence of Cyberspace to the governments of the world:

Cyberspace does not lie within your borders.  Do not think that you can build it, as though it were a public construction project.  You cannot.  It is an act of nature and it grows itself through our collective actions…

You have not engaged in our great and gathering conversation, nor did you create the wealth of our marketplaces…

You are trying to ward off the virus of liberty by erecting guard posts at the frontiers of Cyberspace.  These may keep out the contagion for a small time, but they will not work in a world that will soon be blanketed in bit-bearing media.

The century-long corporate dominance of our marketplaces is at its end.  Together we have built something better which works for all of us.

We have won.

And in the end…

After a brief discussion with a friend today comparing and contrasting the merits of print vs eBooks, I began to re-examine my obsession with record collecting (and with materialism of any form).  I’ve been grappling with both sides of the collectibility coin for years, but the end of a calendar year seems a fitting time to look back and reflect on exactly what I’m doing and at what cost.

My purchasing of printed books has become more selective and refined in the past year (as it has for much of the public with eBook sales still on the rise.)  Statistia.com forcasts that ePub sales will exceed those of printed books by 2017.  This has been my most active year ever for book purchases – the same year I built myself a virtually-departmentalized library of eBooks.

Similarly, my vinyl purchasing has become specialized as well.  2014 has been Innerspace’s most-active for both digital and vinyl acquisitions, with each directly inspiring activity in the other.

107 LPs (accounting for each of the discs in multi-disc box sets) were purchased in the last 5 months alone, making the second half of 2014 our busiest purchasing period in the history of the Innerspace Library.  Actual spending for album acquisition in 2014 (purchases between Jan 13 and Nov 26) totaled $1,140.30, including all spending for overseas shipping and other courier services, with a mean monthly expenditure of $103 for the 11 months of activity.  The vast majority of these purchases were special-edition, limited releases, and original-pressings of milestones from my favorite genres.

Comparatively, 4,138 of my digital album folders were added or modified in 2014, (though this number includes folders in which tag maintenance or restructuring occurred during the last calendar year.)  The increase in digital album “consumption” had a direct impact on my vinyl-ordering activity.

But increasingly, the reasons I’ve used to justify my LP purchases are being eroded by the changing landscape of the FLAC community.

And so I thought I’d take each head-on.

CLAIM 1: Vinyl often features better mastering and production quality than their digital counterparts

The genres I collect, particularly avant-garde, modern classical, ambient, and experimental electronic music have an audiophile fan base dedicated to the digital preservation of these recordings.  Where once fans had to rely on pirate remastering work by Purple Chick and Dr. Ebbetts (among other legendary engineers), the democratization of recording technology has made home-archiving inexpensive and easy without having to chase down shiny black discs.

And for albums previously only available on wax, we have claim #2.

CLAIM 2: You can’t find these recordings anywhere else

Thanks to archival technology, there has developed a large and well-networked community of collectors eager to share their rare vinyl recordings with the rest of the world.  The community has evolved to the point where vinyl-only issued and limited-press recordings are now readily available in the form of community-generated digital lossless archives.  While commercial networks like Spotify offer only a tiny fraction of these recordings (I think there are six) due to their limited commercial interest and costly rights negotiation, actual fans of the music have stepped up to the plate and made the albums available where a commercial market has not.

CLAIM 3: Supporting the artist

This is a moot argument in my specialized case and in the case of those like me.  99% of the albums I buy are used records pressed forty years ago.  The used market has little to no impact on composers and artists, (with a few special exceptions like that of Rodriguez).  One exempt title which comes to mind is Thom Yorke’s Tomorrow’s Modern Boxes LP.  Yorke made headlines when …Boxes became the first album ever to be sold through a commercial torrent channel, demonstrating the viability of the sharing medium.  I saw my purchase of the deluxe LP as a contribution to a cause I supported.  But those two exceptions aside, I don’t think Ludwig or Karlheinz need my money that badly.

CLAIM 4: It’s all about the experience

The experience is one of the outstanding merits of vinyl albums with which digital music cannot compete.  Selecting the album, removing it from its artful gatefold cover, dropping the needle, and so many other elements of the vinyl experience remind us of the value of music in an age of digital gluttony.  The experience is what still motivates me to purchase records, but with greater selectivity than before.

REAL FRIENDS HELP YOU MOVE RECORDS

With the erosion of the above arguments in vinyl’s favor, the reasons against collecting grow larger than ever.

REASON 1: They weigh a ton.

Ask anyone who’s been generous enough to help me move.  Collecting records requires the dedication of space and the enlistment of an army should you ever need to relocate them.  I know it’s part of vinyl’s charm – they are Objets d’art, but they’re admittedly a burden at the same time.

REASON 2: The stuff I’m after costs a fortune.

That $100 a month is over a grand at the end of the year that I could put toward living more comfortably and taking care of my beautiful partner.  I’m grateful that I am able to accommodate the limited disposable income I have for a fulfilling hobby and as a means of social interaction, but greater selectivity may yield a greater reward in the end.  Contrariwise, the only available digital resource for these vinyl-ripped and rare recordings is the file sharing community, which is 100% free.  And there isn’t much that can compare with free.

REASON 3: Accessibility and Organization

What I love most about our Digital Library is that it is meticulously organized and instantly indexable by multiple points of metadata.  With just a few clicks I can export charts and visualizations of library data for my annual reports.  And with 100% of the content on my home server, I can access any track wherever I go.  My unlimited data plan grants me uninhibited access to my content in FLAC without needing to transcode to stay below a corporate-determined threshold of data.  I take several TB of content to work each day, enjoy it on the walk there and back, and DJ my office for the 8 hours in between.

(Oh yes… and 13,000+ albums don’t take up any real estate on the shelf.)

And so…

Moving forward into the new year, I’m going to significantly pull the reins on my vinyl-buying impulses.  I might attempt to quantify my purchasing decisions with a 3-question qualifier before buying (as I’ve a fondness for doing things mathematically.)  There will still be incredible albums here, and there is no reason I can’t talk about a FLAC vinyl-rip and throw up a shot of the LP… (it’ll be our little secret.)

I welcome your thoughts.  Please feel free to share your support for or against this notion.  And I’ll see you in the new year.