My Favorite Variable

Geeking out.

Hello. I'd like to share with you a complex variable in which I take great pride. It's from the Ftrain parser.

A word of context: the Ftrain parser builds complex data structures from a preset XML document, organizing all Ftrain content in a Document->Book->Chapter->Entry structure. This structure uses a combination of arrays, or one-dimensional lists (to preserve order) and hashes, or two-dimensional records (to create records). Unfortunately, I don't know a better way to deal with the problem of XML than with Perl. This will change when I learn more about the Document Object Model, which turns documents into structured, standard information, and XQL, which gives XML database functionality. But I didn't want to wait and learn, or work my way around the buggy Perl DOM and XQL modules.

So here I was. I had an array (Book) which contained hashes. An element in each hash was an array of chapters, which in turn contained hashes, and again contained an element which was an array. In the past, I would have written a function that made another hash, an easier one to work with, one that stood alone, and use that to cross-index the documents. But I don't like redundancy, and I don't like setting, and later undefining, global variables. That's why I built up a large list-of-lists-and-hashes in the first place, right? I wanted to have one large data structure in memory, and access it procedurally. This is, after all, a computer, not a card catalog.

In my program, I needed to find the last member of the entry array in the previous chapter. This piece of information is essential for including the "previous" link on every web page, and to have "previous" work across the boundary of chapters. In my program and my documents, entries are walled in by chapters.

So, in a fever, I wrote (all on one line):

After about 35 tweaks, adjusting brackets and operators and running the script over and over, sometimes fetching nothing, sometimes getting the right answer every other time, I added a final $#, and the script began to fetch the right piece of information, exactly on schedule.

There it was--a perfect "previous entry" link, leaping across chapters. I felt as if I had grown a short notch as a scripter. I had peered into a complicated data structure, a gnarl of interconnected information so intricate that only a theologian could fathom it, and made it perform my desire.

I'm sure there's a way to shorten my hundreds of lines of code into two or three map and grep statements, and make it run in a fiftieth of the time. Guys like Randal Schwartz and Tom Christiansen could rewrite my program in seven minutes and have it do thirty times as much. But--and it's hard to put into words--when I look at that impossibly complicated single variable, it's hard not to feel the smallest burst of pride in understanding, and enjoy the small sense of mastery it yielded.




Ftrain.com is the website of Paul Ford and his pseudonyms. It is showing its age. I'm rewriting the code but it's taking some time.


There is a Facebook group.


You will regret following me on Twitter here.


Enter your email address:

A TinyLetter Email Newsletter

About the author: I've been running this website from 1997. For a living I write stories and essays, program computers, edit things, and help people launch online publications. (LinkedIn). I wrote a novel. I was an editor at Harper's Magazine for five years; then I was a Contributing Editor; now I am a free agent. I was also on NPR's All Things Considered for a while. I still write for The Morning News, and some other places.

If you have any questions for me, I am very accessible by email. You can email me at ford@ftrain.com and ask me things and I will try to answer. Especially if you want to clarify something or write something critical. I am glad to clarify things so that you can disagree more effectively.


Syndicate: RSS1.0, RSS2.0
Links: RSS1.0, RSS2.0


© 1974-2011 Paul Ford


@20, by Paul Ford. Not any kind of eulogy, thanks. And no header image, either. (October 15)

Recent Offsite Work: Code and Prose. As a hobby I write. (January 14)

Rotary Dial. (August 21)

10 Timeframes. (June 20)

Facebook and Instagram: When Your Favorite App Sells Out. (April 10)

Why I Am Leaving the People of the Red Valley. (April 7)

Welcome to the Company. (September 21)

“Facebook and the Epiphanator: An End to Endings?”. Forgot to tell you about this. (July 20)

“The Age of Mechanical Reproduction”. An essay for TheMorningNews.org. (July 11)

Woods+. People call me a lot and say: What is this new thing? You're a nerd. Explain it immediately. (July 10)

Reading Tonight. Reading! (May 25)

Recorded Entertainment #2, by Paul Ford. (May 18)

Recorded Entertainment #1, by Paul Ford. (May 17)

Nanolaw with Daughter. Why privacy mattered. (May 16)

0h30m w/Photoshop, by Paul Ford. It's immediately clear to me now that I'm writing again that I need to come up with some new forms in order to have fun here—so that I can get a rhythm and know what I'm doing. One thing that works for me are time limits; pencils up, pencils down. So: Fridays, write for 30 minutes; edit for 20 minutes max; and go whip up some images if necessary, like the big crappy hand below that's all meaningful and evocative because it's retro and zoomed-in. Post it, and leave it alone. Can I do that every Friday? Yes! Will I? Maybe! But I crave that simple continuity. For today, for absolutely no reason other than that it came unbidden into my brain, the subject will be Photoshop. (Do we have a process? We have a process. It is 11:39 and...) (May 13)

That Shaggy Feeling. Soon, orphans. (May 12)

Antilunchism, by Paul Ford. Snack trams. (May 11)

Tickler File Forever, by Paul Ford. I'll have no one to blame but future me. (May 10)

Time's Inverted Index, by Paul Ford. (1) When robots write history we can get in trouble with our past selves. (2) Search-generated, "false" chrestomathies and the historical fallacy. (May 9)

Bantha Tracks. (May 5)

Tables of Contents