Chris Gammell and Dave Jones' voices span the chasm of thousands of miles each and every week to speak to each other and industry experts about where the field of electronics is moving. Whether it be a late breaking story about a large semiconductor manufacturer, a new piece of must-have test equipment or just talking through recent issues with their circuit designs, Chris and Dave try to make electronics more accessible for the listeners. Most importantly, they try and make the field of electronics more fun. Guests range from advanced hobbyists working on exciting new projects up through C-level executives at a variety of relevant and innovative companies. Tune in to learn more about electronics and then join the conversation! Visit The Amp Hour website for our back catalog of 150+ episodes.
https://theamphour.com
#584 – Software for Rockets with Charles Aylward
Welcome Charles Aylward of Grizzly Peak Systems!
- I heard of Charles from past guest Todd Bailey, they worked together at Astra. Charles is also a member of the Consulting Forum.
- Astra is a “new space” company
- These are non-traditional companies (sometimes startups), that move a bit faster with modern software methods and using more commercially available hardware (non-rad hard parts)
- Past guests in “new space” companies Joris (Hiber), Shaun (Planet), Brent and Bryce (SpaceX, Relativity Space)
- Astra provides a smaller launch vehicle (rocket), which means the ride share equation changes.
- The rocket equation means it could be advantageous to launch a smaller rocket.
- Chris thought of Charles when he thinks of software and hardware rigor.
- How do you know you’re ready to launch?
- Charles was the software engineer for the early launch vehicles.
- Design reviews early on had fewer people but pulled in other people. Todd was working on electronics.
- 6 months to launch
- There’s no dev kit for starting a rocket.
- A rocket is a distributed system
- Multiple clock domains
- Lossy comms
- Charles and Todd talked through some guiding principles
- “No modes”
- “Debug data and test data is first class data”
- “The principle of least surprise”
- “Boot causes no action” (powering on a system causes no action)
- “The system is only hot under control”
- “Only a single source of control at a time”
- “Messages are globally unique”
- “When you’re sending commands to subsystems, you are sending the absolute state”
- Some other (out of order) thoughts around the guiding principles discussion
- Cataloguing all software failures on space systems
- Need to design in wider data channels as a result
- Exposing parts of the RTOS stats
- Safety related
- Default off states
- Pointy side up, fire side down
- Nodes on a network
- Babbling node
- Deterministic system
- Priorities are similar for all systems on a launch vehicle
- What are the constraints of the system?
- Creating a walking skeleton
- Deciding to put ethernet on board
- Desirement
- “Juddson’s Razor”
- Using an RTOS
- Classes of real time contrsaints
- Soft
- Firm
- Hard
- Real Time Systems – Hermann Kopetz
- Hard Real Time Computing Systems – Giorgio C Buttazzo
- PID loops
- So what was it like at Astra?
- Charles joined in 2017, first launch 364 days later
- They launch things like cube satellites (10x10x10 cm)
- The vehicles can go up to 100s of kg
- Still getting new space contracts
- Startups
- “Rigor gets a bad rap because of agile”, but there’s a crossover point for investing in the setup of new systems.
- In consulting, there is often pressure is high to get first proto out quickly
- Azonenberg’s checklist on github
- “What happens if the test fails?” You need to have recovery time baked into a schedule and a plan
- Contact
- Freelance info
- Other links, sent from Charles after the recording
- I left Astra before this particular launch, but Scott Manley has a good/fun video on what happens when your systems don’t “fail fast” but instead “fail operational”
- You asked where one can pick up some of this stuff (regarding rigor etc). A lot of my early thinking on robust software was formed by Robert Rasmussen from JPL. He has published a lot but by far the best synopsis on thinking about dealing with inevitable errors (fault tolerance) as not a special case, is in this paper. This basically ties right in with the “observability” thing we discussed.
- Another good foundation for coding standards is the JPL “powers of ten” paper. There are things I disagree with, and outright reject, e.g. pointers can’t be used safely, but, like the principles we discussed on the show, it’s not that there’s some prescriptive golden list, it’s just that you want to build out a complete list like this of your own. Pulling what you think makes sense from JPL P10 or MISRA can really help.
- The best way to contact me is probably twitter.
Click image for link to Charles’ site with explanation