yellowpigs.net
Sara Smollett, Hyperbolic Consulting LLC
ssmollett@hyperbolicllc.com
- 25+ years Linux admin (Debian GNU/Linux, Ubuntu, RedHat/Fedora, etc)
- 15+ years Google Site Reliability Engineer
- 10+ years Solaris & BSD admin
- 5+ years engineering manager
- Current preferred programming language: Python
- Other languages I may (or may not) remember: C, Bash, Golang, Perl, JavaScript, PHP, Scheme/Racket, Java, Ruby, SML
Employment history:
Google, Inc., Mountain View, CA (Oct 2005-Aug 2011, Sep 2012-Jun 2023):
- SRE Manager II, Site Reliability Engineering
(previously Senior Site Reliability Engineer)
- Provided reliability consulting for development teams re-architecting large-scale services (e.g., GMail migration to Spanner, refactoring of Google Contacts infrastructure), and designing, launching, and productionizing new features & services (e.g., Cloud Spanner, Google Calendar mobile apps), as well as sunsetting legacy services.
- Identified & added redundancy for single points of failure and engineered for fault tolerance & graceful degradation in distributed systems; defined & documented incident response & disaster response procedures, including data recovery; led disaster recovery training & drills.
- Performed regular oncall duties for critical user-facing high-QPS high-availability services, as well as essential internal systems (e.g., Google Accounts, Google Calendar, intrusion detection pipeline, HR & finance applications); responded to numerous significant incidents, including writing postmortems & driving long-term followup; developed extensive custom availability & performance monitoring alerts & dashboards; led pager-load reduction efforts.
- Simplified service architecture and automated processes (e.g., release management, backup restores) to improve reliability and reduce toil & cognitive load.
- Significantly improved resource utilization & usage, lowering service costs; performed capacity planning, loadtesting, and service turnups in new datacenters/regions.
- Most of the tech stack I used at Google is proprietary and does not exist outside of Google (although many have external equivalents, e.g., GCP, Kubernetes), but in the earlier days I did use things known to the outside world, e.g.: configured Netscaler load balancers; set up & administered MySQL and PostgreSQL multi-tier replication topologies and automated failover; automated Linux server/services deployments using Bash, Puppet, and creating .deb packages.
- Managed and/or tech led teams of 6-12 engineers across multiple sites/continents: drafted roadmaps, prioritized & assigned project work; coached individuals through onboarding & ongoing career growth; built new teams & merged existing teams; mentored new managers. (2014-2023)
- Local doodle maintainer for google.co.vi domain; initiated custom text on the local homepage commemorating VI holidays. (2019-2023)
- Helped draft AI Principles policy. (2018)
- Co-authored a chapter of The Site Reliability Workbook (Chapter 8: On-Call). (2018)
- Obtained defensive publication for automatic event scheduling based on weather conditions. (2017)
- Served as an SRE hiring committee member and staffing allocations committee chair. (2013-2019)
- Served as a member of the core disaster recovery testing team. (2013)
The Wikimedia Foundation, San Francisco, CA (Dec 2011-Sep 2012):
- Operations Engineer (part-time contractor)
- Automated generation & deployment of Ganglia monitoring configs using Python & Puppet.
Woods Hole Oceanographic Institution, Woods Hole, MA (Nov 2003-Sep 2005):
- Information Systems Associate II, CIS Networking
- Deployed, modified, and maintained a network registration system based on CMU NetReg
(using Perl, Apache2, MySQL 4, BIND 9, DHCPd 3, Squid, Nessus, CiscoWorks) and
developed additional Perl CGI web app and associated tooling & documentation to assist in migration.
- Migrated people/account information from Netscape LDAP to OpenLDAP
(using eduPerson and custom schemas).
- Assisted with deployment of VoIP (Asterisk/SIP) services.
Simon's Rock College of Bard, Great Barrington, MA (Feb 1998-May 2000, May 2000-Oct 2003):
- Lead Systems & Network Administrator, CMS
(previously student Networking & Computer Assistant)
- Linux: Installed and maintained production Debian GNU/Linux servers with ~1000
user accounts and numerous services including web services (Apache, Squid),
database & directory services (MySQL, PostgreSQL, OpenLDAP), mail services
(Postfix, Courier, Mailman, SpamAssassin, ClamAV), and DNS & DHCP services
(BIND, DHCPd).
- Other operating systems: Shared responsibility for Mac OS X, Windows 2000, Solaris,
and AIX servers; set up and performed troubleshooting for Mac and Windows
workstations & desktops.
- Scripting: Wrote Perl and Bash scripts to automate systems and network
administration tasks (account management, log parsing, backups, service
monitoring, etc).
- Networking & security: Configured Cisco & Linux routers & firewalls
(iptables, ipchains); performed bandwidth monitoring & traffic shaping
(iptraf, ntop, mrtg, libpcap) and security & vulnerability scanning (nessus,
snort, argus); installed network switches, hubs, ST/SC fiber & Cat5 ethernet.
- Web development: Created and maintained websites (HTML, CSS, PHP, Perl, Python, PostgreSQL).
- Hardware: Built, upgraded, and repaired PCs & Macs.
- Teaching: Taught an undergraduate course on Linux.
Stanford University, Stanford, CA (Jun 1999-Nov 1999):
- Networking Intern, MedIT and CIS
- Installed and maintained network switches (3Com, Asante, ATI, Cisco, Extreme, HP).
- Provided Mac and Windows desktop, Apple hardware, and networking support.
- Performed routine systems administration tasks (Linux, Solaris).
Other experience:
Diversity & inclusion:
- Google ERG lead (2017-2023)
- Disability resources: Created & ran a disability awareness course at Google [external version] and
co-created an online disability training for managers [external version], as well as a website of related accessibility & inclusion materials. (2016-2022)
- Event hosting: Authored guidelines for hosting inclusive offsites and co-authored the AdaCamp toolkit for organizing conferences. (2014-2015)
- Bias training: Co-initiated Google's Bias Busting workshop [external version]. (2013-2016)
- Hiring: Advised on writing job descriptions to appeal to a more diverse pool of applicants, as well as training for interviewers and recruiters. (2011-2021)
Volunteer:
Academic:
- San Jose State University: certificate in Computational Linguistics (Natural Language Processing) (2011-2012)
- Tufts University: M.A. Philosophy (Philosophy & History of Science & Technology) (2001-2005)
- Simon's Rock College of Bard: B.A. Liberal Arts (Math & Computer Science), magna cum laude (1996-2000)
- Hampshire College Summer Studies in Mathematics: student (summer 1994, summer 1995)
- Antilles School: National Merit Scholar, MathCounts Nationals competitor
- More detailed academic resume
References:
- References available upon request
- LinkedIn