Script for migrating NetInfo system accounts to OpenDirectory accounts

This is going to be above the scope of most “power users” or newcomers to shell operations in OS X, but the following is a zsh script for pulling local system users in NetInfo and moving them into shiny new OpenDirectory accounts. If you don’t know what any of that means, you probably shouldn’t be messing with this. If you do, enjoy.

for i in `cat /path/to/shadowed/password/dump/users`
do
user=`echo $i|cut -d: -f1`
# if [ `nidump passwd .|grep $user|wc -l` -lt 1 ]; then
if [ `$DSCL . -list /Users |grep $user|wc -l` -lt 1 ]; then
#get the rest of the user settings
shell=`echo $i|cut -d: -f10`
realname=`echo $i|cut -d: -f8`
uniqueid=`echo $i|cut -d: -f3`
primarygroupid=`echo $i|cut -d: -f4`
homedir=`echo $i|cut -d: -f9`
passwdhash=`echo $i|cut -d: -f2`
# echo $i|niload -m passwd .
#create user in local OpenDirectory
$DSCL . -create /Users/$user
$DSCL . -append /Users/$user UserShell $shell
$DSCL . -append /Users/$user RealName "$realname"
$DSCL . -append /Users/$user UniqueID $uniqueid
$DSCL . -append /Users/$user PrimaryGroupID $primarygroupid
$DSCL . -append /Users/$user NFSHomeDirectory $homedir
$DSCL . -append /Users/$user AuthenticationAuthority ";basic;"
$DSCL . -append /Users/$user Password "$passwdhash"
# Below would use more secure SHA1 password hash, but requires some setup work
# Shadow password files would need to be stored in /path/to/shadowed/password/dump/shadow directory
# with name in form of $user.shadow (one per user)
#$DSCL . -append /Users/$user AuthenticationAuthority ";ShadowHash;"
#$DSCL . -append /Users/$user Password "*"
#keyfile=`$DSCL . -read /Users/$user | grep -i generateduid | cut -d' ' -f2`
#cp /path/to/shadowed/password/dump/shadow/$user.shadow /var/db/shadow/hash/$keyfile
mkdir -p /Users/$user
/usr/sbin/chown $user /Users/$user
logger -p local1.info "$user has been added to local OpenDirectory accounts"
fi
done

for i in `cat /path/to/shadowed/password/dump/remove`
do
user=`echo $i|cut -d: -f1`
if [ `$DSCL . -list /Users |grep $user|wc -l` -gt 0 ]; then
#niutil -destroy . /users/$user
dscl . -delete /Users/$user
rm -r /Users/$user
logger -p local1.info "$user has been removed from local OpenDirectory accounts"
fi
done

rsync is your friend.

Say you need to keep multiple copies of a directory full of data on multiple computers, so that all of them have an up-to-date bit-for-bit copy of the same data. For me, the prime example of this is my mp3 library. I maintain the master copy of the library on my desktop computer, and I want at least one or two entire backup copies of that library on other computers, so that if my hard drive crashes, I’ve got something to recover from.

This process is called “making backups.” Backups are your friend. And the perfect UNIX tool for this particular task is rsync. Rsync is also your friend.

rsync -rav /path/to/your/master/copy/of/Music/ username@otherhost:/path/to/your/backup/copy/of/Music

You can also get much fancier with this, and exclude particular file types if, say, you only want audio files, and no copies of movies. Simply add an “--exclude '*.m4v' (or whatever suffix you want to exclude) to the command, and boom, rsync keeps everything except m4v files up to date between the two.

rsync -rav --exclude '*.m4v' /path/to/your/Music/ username@otherhost:/path/to/backup/of/your/Music

Naturally, you’d want to set up public keys on the host in question, so you don’t have to type your password all the time, but we’ll get to that later. For now, you can type the command, substituting the appropriate paths for the source and destination you want to copy from/to, type the password that one time, and then marvel at the automated syncing.

As is, this is only going to execute when you manually run the command. However, wrap it in a shell script, and add a crontab entry for it, and presto, you’ve got nightly backups going on. And yeah, I’ll probably write up a brief “crontab for beginners” here, as well, in the event you’ve never used that. Or you could google it for yourself. For now, here’s an example of what the output will look like in your shell:

mns$ rsync -rav --exclude '*.m4v' --exclude '*.mov' /Volumes/shellfish/Music/ mns@myserveraddress:/Volumes/ashurbanipal/Music
building file list ... done
./
Chris Bell/
Chris Bell/I Am the Cosmos (Deluxe Version)/
Compilations/Essential Irish Drinking Songs & Sing Alongs - Whiskey In the Jar/
Compilations/Sea Shanties/
Compilations/The Essential Kris Kristofferson/
Danu/Up In the Air/
Downloads/
Downloads/You and Your Sister (Single Version) _ I Am the Cosmos (Deluxe Version) _ Chris Bell.tmp/
Flying Column/Four Green Fields/
Home Videos/
Luke Kelly/The Best Of Luke Kelly/1-04 The Gartan Mother's Lullaby.m4a
Luke Kelly/The Best Of Luke Kelly/1-06 Alabama '58.m4a
Luke Kelly/The Best Of Luke Kelly/2-06 For What Died the Sons of Roisin.m4a
Luke Kelly/The Best Of Luke Kelly/2-08 The Town I Loved So Well.m4a
Luke Kelly/The Best Of Luke Kelly/2-10 Maids When You're Young Never Wed an Old Man.m4a
Matthew N. Sharp/
Podcasts/
Podcasts/Ridiculous Dialogue/34 - Swimming in Denim.mp3
Podcasts/Ridiculous Dialogue/35 - Milk Sandwiches.mp3
Podcasts/Ridiculous Dialogue/36 - The General Leak.mp3
Ronnie Drew/The Best Of Ronnie Drew/1-02 Donegal Danny.m4a
Seamus Ennis/Forty Years of Irish Piping/17 Piper of the Embers _ Down the Back Lane _ Sixpenny Money _ Paudeen O'Rafferty.m4a
Seawolves/Seawolves/02 Chemin de Francois.m4a
Seawolves/Seawolves/10 Gaspe Reel.m4a
Seawolves/Seawolves/11 John Kanakanaka.m4a
Shane MacGowan/The Snake/02 Nancy Whiskey.m4a
Shane MacGowan/The Snake/03 The Song With No Name.m4a
Shane MacGowan/The Snake/16 Donegal Express.m4a
The Dubliners/50 Years/09 Raglan Road (feat. Luke Kelly).m4a
The Dubliners/50 Years/14 Scorn Not His Simplicity (feat. Luke Kelly).m4a
The Dubliners/50 Years/43 McAlpine's Fusiliers (feat. Ronnie Drew) [Live].m4a
The Men of No Property/This Is Free Belfast!/13 The Bogside Man.m4a
The Men of No Property/This Is Free Belfast!/15 Ballymurphy.m4a
[long list of files deleted]
Willie Nelson/The Essential Willie Nelson/2-19 One Time Too Many (with Steven Tyler).m4a

sent 522820836 bytes received 498124 bytes 6666483.57 bytes/sec
total size is 492744977453 speedup is 941.58
mns$

OS X “Spotlight” from a shell

Ever lost something on your Macintosh that you’re sure you saved someplace? Can’t find a particular email on a subject, but you remember a keyword that might help locate that email? Spotlight is OS X’s built-in operating system function that allows you to search an index of the contents of your computer, and quickly find that old email, or a file, based on a keyword search. To invoke the GUI version, you need only hit the Command-Space Bar combination, and up pops the dialog box waiting for your keyword.

But you can also do this from a shell, from within Terminal.app. While the usefulness of that feature on a computer you’re currently using might not be that great, it does come in handy if you have multiple Macintosh machines, and you’re sure the file is on your other computer, but you really don’t feel like getting up and walking over to it to check. Simple ssh over to that machine, and issue the following command:

mdfind -name <whateverYou’reSearchingFor>

An example; I wanted to look at my rsync script that I use to make sure my entire music library gets copied to multiple hosts inside my network, but that would have required walking to the office to use Spotlight there in order to search for “rsync.functional”, since I knew that was the name of the script. Instead, I did the following:

tritium:~ mns$ ssh radium.local

Last login: Fri Oct 30 14:29:28 2015 from [tritium.local]

radium:~ mns$ mdfind -name rsync.functional

/Users/mns/Documents/rsync.functional

Now, if I had only known that the file had “rsync” in the title, but didn’t know the whole title, I could have used ‘mdfind -name rsync‘, and it would have returned a larger list of files containing “rsync” in their filenames. There are also a host of other command line options for the mdfind utility, which, if you’re curious, can be seen by taking a look at the man page for it. Issue “man mdfind“, and it will explain a great deal more.

Introducing a new category for this “blog” thing: Nerdery

I’ve generally hesitated to write up technical tips or how-to entries, largely as the result of being restricted from writing about certain technologies for NDA reasons for a number of years. That said, I’ve still been asked countless times to help friends or family with technical issues, and as a result, I’ve written plenty of how-to documents or handed out quick lessons in technical subjects for the benefit of people who know me, and largely because while it’s simple enough to fix one person’s particular problem, it’s even more rewarding to teach them how to solve those problems for themselves.

Given my work experience as a UNIX system administrator for almost 20 years, I’ve acquired a certain body of technical knowledge that I am no longer restricted from writing about. Further, my extensive experience with Mac OS X Server and Mac OS X, running the entire duration of the existence of those operating systems, puts me in a rare category of UNIX admins. Because of this experience, I’m going to start writing more technical things here, on top of the usual expository dreck on subjects I’ve found interesting for years.

Some of these tech-oriented pieces will be geared towards newbies or would-be “power users”, some will end up being more complex, and better-suited for experienced systems administrators. I’m not going to try to define (and subsequently restrict) the content at this point; I’m just going to start knocking entries out, and we’ll see where it goes. Fortunately for all of us, you, the reader, still retain the ability to skip reading articles you don’t want or need to read. I encourage you to exercise that discretion accordingly, and if you happen to find some of this useful, well, huzzah! If nothing else, it will serve as a repository of material from which I can send links to the same said friends and families, so that I don’t have to re-write the same how-to articles tailored to each individual problem.

And I promise that this article will be the least useful of all of them, since it is of an administrative (in the non-system administration sense) nature, and therefore provides no actual technical knowledge, whatsoever.

Heads Buried in Sand

Daniel Denvir wrote an article in Salon recently about what he calls the “shocking, Orwellian rise of “school resource officers””, bemoaning his imagined rise of a “police state”. The article, predictably enough for Salon, does its level best to stoke the fires of racial strife, jumping to a series of conclusions without waiting for evidence or context in the heavily circulated video case of school resource officer Ben Fields in South Carolina forcibly removing a student from a chair she apparently refused to vacate when ordered to do so. This article, posing as a piece of journalism, is entirely an advocacy opinion piece of the type one would expect from Salon. If it were journalism, it would be highly irresponsible journalism, as a key element of any work of journalism would be the desire to flesh out any and all pertinent facts before coming to any particular conclusion.

The incident in question happened too recently for anyone in the public to be expected to have all the facts. The investigation, itself, is not complete, though that does not stop opinion “journalists” from using the incident as evidence to support their preconceived ideas; in fact, it serves as a convenient excuse to run such half-baked pieces, in an effort to capitalize on public attention. Further, and to avoid any mincing of words, the financial motivation for publishing these pieces is entirely based on the corporate sponsors’ desire to sell advertisements that surround the prose written by pundits, while the public still cares enough to read about the event, in an environment where the collective attention span is two to three days, maximum.

When you take this incident in the context of school security, you can expand the subject to include school shootings, as well as the significantly less published events wherein school authority figures have been attacked, physically, by students at said facilities. The reality of why there are an increasing number of “school resource officers” is because administrative organizations in charge of schools, and therefore school security, realize there is an increasing problem that they need to address, and that the best way to address these issues is to include the on-campus presence of personnel tasked with, and ideally properly trained for, handling these occurrences.

Denvir decries the response of this SRO as evidence of “racism”, implying that schools are creating a “school to prison pipeline” for certain members of the student body.

“For poor children of color, the mouth of the school-to-prison pipeline is manned by police officers who have in recent decades proliferated in districts nationwide. The mass deployment of schools cops, commonly referred to as “school resource officers,” has been made without careful thought or research. And it has produced horrible outcomes.”

He offers no proof that there has been no “careful thought or research”, nor that it “has produced horrible outcomes”, but he writes both things anyway, which then become claims that other half-assed journalists, or bloggers, will repeat, and if we know anything, we know that repeating half-truths results in lazy readers embracing these half-truths as truths.

Reality, of course, does not care what any number of “journalists” or bloggers believe. And reality is that a small portion of the student body today apparently feels emboldened enough to physically attack staff members of their institutions for whatever reasons they feel aggrieved by. Reality is that an even smaller number of students, for whatever reasons, have felt aggrieved enough to enter schools with weapons, intending to inflict as many casualties as possible, and they’ve gotten away with it far too often, largely because these schools have not had the means to stop them.

School Resource Officers are the actual solution to this problem. And yet, when one suggests that fact to the rabble that comprise the anti-gun movement, the immediate outcry is that this is a “disaster waiting to happen”. They fantasize that the mere presence of trained, armed individuals in a school will result in a non-stop “wild west” shootout. They claim that our schools don’t need armed individuals to defend the unarmed student body, or the unarmed staff of these schools, and they exclaim that they don’t want to live in a world where any of this is necessary.

But they do live in that world. We all live in that world. Their desire to “get rid of all the guns” is unrealistic. There are already 350,000,000 guns, legally owned, in this country. On top of that, there are an unknown number of illegally procured guns. And there are miscreants in this world whose sole purpose, usually in pursuit of notoriety via mass media coverage, is to inflict damage and pain in as large an amount as possible. You may not like the fact that this is your world, but it is your world, regardless.

The analogy that is most apropos is security as it is applied in Israel. Realizing that their schools were a target for miscreants with the intent to do harm, Israel addressed the problem, rather than wishing they didn’t have the problem in the first place. Schools are surrounded by layers of perimeter security, and staffed by trained, armed “school resource officers”. As a result, school shootings in Israel are minimal, in comparison to other nations of similar circumstances. One may argue that the threats faced in Israel are not directly comparable to the threats of maladjusted young adult shooters in the United States, but actually such a comparison is fairly accurate. While the motives behind attacks may differ slightly (eg: religious war vs. maladjusted desire for notoriety), the situations themselves are very similar.

The question, then, is whether we, as a people, are going to continue to deny we have a security issue, or refuse to act because it means accepting the world we live in is not the world we wish we lived in, or whether we will accept that in order to prevent tragedies, we are willing to take the obvious, efficient, and right courses of action in order to prevent them. Will we do what we need to do, or will we bury our heads in the sand, and hope the insanity stops?

What is entirely not useful, however, is posturing political advocacy pieces from the likes of people like Denvir, decrying “racism” or “police brutality” at every corner. Denvir has no solutions. I’m not going try to put myself inside his head to imagine what motivation he may have for wanting things to stay the same, but he has contributed nothing towards building a solution, and is but a roadblock in its path, and, as such, he should be embarrassed for writing what he has written. But I’m sure he won’t be. He works at Salon, after all, and no doubt received more than a few “atta-boys” for restating the Groupthink conclusions they hired him to write. After all, he submitted it on time, and before the Collective Attention Span had averted its eyes to the Next Approved Outrage, so Salon was able to garner plenty of social media traffic, increasing the marketed value of impressions for their brand of web propaganda.

An Ubiquitous Cultural Shift in Focus

I started working at Apple in 2006, a year or so before the release of the iPhone. While there were certainly “smart phones” around prior to the existence of the iPhone, they were significantly less prominent. I had owned devices from Blackberry and Palm, and was fascinated by the technological developments being made with faster, smaller processors, and increasingly more powerful phones. Being able to check email from anywhere was relatively new, and as the various phone carriers upgraded their network speeds, browsing the web for information became easier and less painful. But once the iPhone dropped in 2007, then Android phones started appearing, as well, the “smart phone” took off. It was no longer just tech-obsessed “early adopters” who had these things; it was everyone. It was your mother, your grandmother, your neighbors, your barista, and eventually, your elementary school students. Over time, most cell phone users moved to “smart phones” of one type or another, and over that same period of time, the social behavior pertaining to the use of these devices changed, as well.

At first, it was less noticeable, in that you might see someone on a public street stopping to type something to someone with their thumbs on their smart phone from time to time. Then you’d see people driving, with one hand on the wheel, and their other hand on that smart phone, eyes focused on whatever it was that had captured their interest. Then you’d see a table full of people out at a restaurant, having a group dinner, all of them focused exclusively on the content their phones were delivering to them. You’d see people actively walking down the street, not looking in front of them, reading this or that on that tiny screen, as if what was happening on Twitter or Facebook was infinitely more interesting than the world they were physically moving through. You’d see people ordering food, then taking pictures of that food, rather than eating it, in order to broadcast said photographic evidence of the meal they’d ordered to the rest of the Internet. You’d go to a concert, and instead of seeing hundreds of hands holding up disposable Bic lighters, those same hands were holding up iPhones or Samsung Galaxies to capture crappy video clips of whatever band they’d paid thirty bucks to see. Rather than watching, listening, and enjoying the experience, they were trying to capture a small bit of that experience, not so much because they wanted to be able to remember the experience, but because they wanted to “net brag” to their friends and associates via social media that they were watching Band X right now, while the friends and associates were not. But they weren’t actually watching Band X. They were, instead, engaged in narcissistic self-promotion.

Sociologists have begun to explore the cultural and social changes that the greater prevalence of mobile network connectivity and social media have brought us, though there is still much more studying to be done. And as these studies are being explored, new social and cultural changes continue to develop. Much of these changes have too much momentum to effectively be thwarted at this point, as cultural buy-in applies societal pressure to large masses of people to engage in the same behavior, lest they be left behind. When communication was predominantly face-to-face, behavior was obviously different than it became once communication moved to electronic forms, with the introduction of increasingly large amounts of passive-aggressive or aggressive-aggressive behavior present, possibly due to the bravery provided by a certain sense of anonymity. That’s neither here nor there for purposes of what I’m getting at; we are, as a society, increasingly reliant upon surprisingly fragile layers of technology for tasks from simple navigation, to communicating with each other, to making decisions about life, about paths, or about banal consumer choices. Looking at this dependency from a security analysis standpoint, one cannot help but notice the potential flaws that could prove disastrous to a society reliant upon these layers of technology. It can all fall apart. It can collapse under a cascading failure of any number of factors; electrical power, cabled network infrastructure, and cellular infrastructure all depend on one another at this point. Nature, or bad actors (which, arguably, could still be considered “nature”, given than human beings are still a part of nature, whether we fancy ourselves outside or above it or not), or catastrophic accidents could easily collapse one of the underlying pillars, and depending on the scope, human-planned recovery may not be automatic, easy, or even possible in that event. What remains to be seen is the actual effect of this infrastructure collapsing on society as a whole. For localized areas that aren’t entirely reliant upon it, the effects may be minimal. But for large urban centers, it could be devastating. And large groups of humans aren’t known for their reasoned, measured responses to major tragedies.

Forehead, Meet Lamppost

There was a moment, way back in those heady days of 2011 or so, that I was walking from one place to another in the Lower Haight neighborhood of San Francisco. Unsure of my destination’s physical address, I unlocked my phone and pulled up the map application, and began typing the name of it into the search field, at which point I felt a solid “thud” against my nose and forehead. I looked up from my magical electronic navigation device to see a light post that I had walked straight into, because my focus had been on the magical electronic navigation device rather than, say, where I was actually going. I felt the wetness of blood forming streaks from my forehead, down my temple, and into the wells of my eye, and wondered, at first, if I might need stitches. I blotted the blood away as best I could, but just as soon as I would finish blotting, the wound would pump out more blood to replace it, and so I ended up just holding the paper towel over my right eye, to at least keep the blood out of the eye, itself.

At that point, I decided it was time to cut the shopping trip short, and return home to tend to my fresh wound. Changing course, I tried to flag every taxi I saw, but none of them were interested in picking up an actively-bleeding fare. A couple of them slowed down and pulled over, but once they caught a closer glimpse of my bloodied face, they’d pull away and leave me behind. The second one actually hit the auto-lock on his doors before doing so, a fact which was both audible and visible to me, since I had managed to get within about two feet of his cab. My legs still worked, so I kept walking towards home, and when I got there, I gained a better understanding of why it had been so difficult to hail a taxi; the right side of my face, neck, and t-shirt were absolutely covered in blood. I wouldn’t have picked me up if I was a driver, either, if for no better reason than not wanting to have to clean up my cab afterward.

For years, I’ve prided myself on having a better than average sense of “situational awareness”, yet here I was, covered in my own caking, dried blood, all because I had been looking at my handheld glass screen instead of the unmoving mass of steel and aluminum on the sidewalk in front of me. I cleaned myself up, slathered a large bandage in Neosporin, and vowed to never let something like that happen again. “Eyes up, real world shit,” from now on.