I’m just adding some random career thoughts after a few years as a programmer. Just want to say, my opinions are my own and not the views of my employer.
Before the Job
During the Job
Example 1 (excel at your core job functionalities first):
I was talking to a newer engineer that wanted to improve the team’s onboarding process. No one stopped him from updating documentation, but I doubt that was promotable work since it was difficult to measure the impact of documentation and he didn’t have much code to show. You need to cover your core skills first for your level (e.g. L1 == coding for a software engineer).
Example 2 (choose your own project):
We currently have a project that’s been running 2 months with 3 engineers and is still months from finishing. The app only has about 20 daily users with about 30 sessions a day. The main impact is we don’t have to run an operational task (that takes 10 minutes after business hours) once every 30 days. It’s low impact work so even if you did a great job technically, it doesn’t mean much; avoid that project and propose higher impact projects that the team can work on instead.
Switching Jobs
Problem: We have a RDS table ‘history’ that tracks every change data capture on the entire database. That means Table A or B makes an update, we write that update to the ‘history’ table. Our database suffers from brownouts because of the high load on this table (e.g. 50% of writes)
Solution: Moved the ‘history’ table outside of the RDS using Debezium (kafka connect with a mysql source connector) that writes to Kafka topics (1 table == 1 topic). Created data pipeline to make the data from kafka queryable by having another Kafka Connect (Kafka Source to S3 Sink), a Lambda tranform to another S3 Sink, and Athena on top
Probably due to recency bias, but the project before my current project was basically Google Analytics (i.e. Event Tracking), e.g. you click on a button, it fires an event saying you clicked on X button and use that data to determine user behavior.
The problem was that even naming events is tough. Fixing it when you’ve been doing it wrong (no plan) for multiple years (e.g. people named events ‘ButtonClick’, ‘ClickedButton’, ‘ButtonClicked’ with different properties / no standards) will take a while (e.g. working across multi-disciplinary teams including product engineering, business intelligence, data science, data platform, etc) and a lot of alignment. Leadership thinks that the company is a startup and can fix issues in weeks, but doesn’t realize that it’s a cultural problem as well as a technical problem that will take years to fix for a $XX Billion company (i.e. not a startup anymore).
When teams work on a feature, some don’t think “What’s a way I can track if this is successful or not; how will I know if I tune X or Y know if it’s helping or making it worse?”. Without that data, you’re relying on really lagging indicators (e.g. people quitting your app).
The teams that do think about data, the next question is “How do I do this?”. This requires a LOT of alignment, especially if teams are working across different segments of the product.