I liked this episode for the same reason I liked the one with Henry Golding. This episode covers leadership approaches for QA transformation through Gary’s organizational change experience:
Key takeaway: His approach of avoiding buzzwords, continuous experimentation, and building internal capability closely mirrors successful QA team transformation strategies.
I trained my team myself, had weekly one on ones and there were no consultants. Each week I reviewed what I learnt with my leads and we kept trying new experiments.
I can’t say there was a vision or strategy handed to me by the execs. At most the director who put me in that position was very supportive. When he was let go by the new VP, the new director (my boss’s boss) provided no direction, just the usual exhortation to do better. I was asked by the new directors what the plan was because they were looking for an improvement programme. I said there wasn’t one, that’d we’d just continually improve opportunistically. I’m going to assume they left me alone to do so initially because they must have thought I’d be sure to fail with no budget.
During my time on the QA team, Thoughtworks consultants were also brought in but they were basically given the run around. After that VP left and then the 3rd VP took over, she had an internal DevOps consultant who wasn’t technical and that went nowhere.
I too avoid the terms Agile or DevOps because of the pre-conceived notions people have of what those words mean and prepared arguments for why that won’t work here. My approach was to address common cause variation around time to fix a failing test through PDSA cycles.
I think when we start, software development looks more like a manufacturing engineering problem. I think companies just set it up this way because they think it’s a manufacturing problem. However by the end of a transformation, we should have something that looks like a design engineering problem. If everything done by someone manually could be converted into a self-serve platform, then you don’t have all the hand-offs that make it look like manufacturing. For example, instead of handing off the code to operations, we have services that allow developers to do so themselves. I wanted the same thing but for QA tests; I described my approach to doing so by developing a platform here.
I tried to implement Dr Deming’s 14 points for management. The system was a legacy tightly coupled system where you have to release the software together. The single sign-off required the work of almost 60 QA team members. The biggest sources of inefficiency were across the team because of the waterfall process and big batches of work. I could influence people within my team and the COBOL dev team but it was hard to do across the other teams.
I communicated to my team what my vision was for the team and how we were going to get there. Given the size of the QA team and the task of reducing the cost, my salary was pretty much the budget for improvement. Any talk of consultants/contractors meant that people’s jobs would go to India so that wasn’t an option for my team in Canada. Also if I told people what I had in mind, I’m pretty sure I’d have been told no which is why I adopted the gravity slingshot strategy. If you’re familiar with manufacturing and Goldratt then my post about creating pull explains what metrics I used to measure our progress.
Because of the industrialization of agile, I took a buzzword free approach. As explained in Start With Why I only explained what TDD/BDD was to motivate my team. I showed my team the manifesto for agile software development and explained the Poppendiecks work on lean software development.
In my mind I tried to apply my very limited understanding of Dr Deming’s work to the process of inspection; basically to do point 3 and cease dependence. It’s why I adopted the term Deming Driven Testing to describe that approach
In my case, my QA team was basically 4 QA teams of no more than 15 people each. The main areas of friction in the span of my control were between those 4 teams when they had to co-ordinate to do system testing The other areas of friction were the development and BSA teams. Given I had read everything the Poppendiecks had put out, I leaned heavily on manufacturing process improvement.
I had previously worked with the COBOL developers as a developer on the Java/PLSQL parts of the system. That and the relationship my COBOL testers had with them gave us some influence in the way they work. To make it easier for developers to invest their time I fostered a culture of easy ownership.
I agree that most people I’ve worked with have never seen what good software development looks like. Even folks who think SCRUM works without TDD do so because they work on such small projects that even waterfall would be fine. The director who officially took over from the one who moved me to the QA team thought he was doing just fine. He could find nothing wrong with how he did his job. At most he thought QA needed to do better because you know, quality is always inspected in. Him having a PMO agressively defend waterfall and working with big batches wasn’t the problem ;)
Just as people would argue with him till the cows came home about TBD, I avoided buzzwords. My recommended approach is simple, use a process control chart to study the time it takes to resolve a test failure, keep looking for common cause variation and remove it through PDSA. The reason for developing the testing platform was so that developers could organically opt into new ways of working that progressively gave them more feedback earlier. Before that though, my testers had to change as well, which is why we had our internal enabling team.
I also agree with not going someplace and just re-organizing everything into small teams or re-architecting into a loosely couple architecture. Compared to what was in the DevOps handbook, in that org, it’s so far from how they’re doing things, they could easily find ways to push-back. In the end my large QA team worked with COBOL developers from another manager’s team; they didn’t have to be combined into one.
This system didn’t need to become a collection of micro-services. I think in more cases than people think, you can do with a modular monolith or even a monolith if you have earlier feedback. You make sure you have feedback to make changes in small steps whatever the size of the system. You need to work in a way to make sure your system is in a releasable state at all times. Had the momentum continued, we would have been able to do Continuous Delivery with COBOL. This was possible with this particular system because almost all functionality was controlled by feature flags.
Because the architecture was outside of my scope of control and the architecture team was the ivory tower type, waiting for micro-services to help solve testing challenges wasn’t an option. There was lots of low hanging fruit that I could go after first simply by identifying and removing waste. In fact my whole strategy didn’t need any team outside my own to change. Let’s say the COBOL developers didn’t change. Then to reduce the waste of incorrect test cases, I’d have invested in model based testing which lets you generate thousands of test cases in a few minutes.
When I joined the QA team, I too was trying to answer how to move forward in an org that has lots of manual testing. Fully automating the system tests wasn’t done and instead we focused on unit tests for the most complex component. We got there by working in small batches and working with developers to prevent defects rather than automate inspection for them. Eventually we’d be able to remove manual testing as a quality gate.
I think AI can help accelerate that process.
This episode covers key areas relevant to QA transformation through Henry Golding’s Minecraft development experience:
Key takeaway: Henry’s experiences closely parallel QA team transformation - focusing on earlier feedback, easier developer workflows, and organic adoption over imposed processes.
I was a developer and also found myself in a QA team. I joined with the intention of making the lives of developers easier by changing the QA team to focus on earlier, not faster feedback. When I started on the QA team, I knew nothing about QA processes. Later I found that half of which appeared to be testing theatre.
The benefit adjudication engine was fairly complex. When a mistake was made at a low level, it would take a long time for the QA team to identify where the issue was so that they could assign the bug to the right person. Then the developer had to setup all the data to try and test it which would take them a while. What my QA team did for them in the end was make it easier for them to recreate the setup data in their dev environment. If a defect was found later on in the QA environment, it was more likely that the source was one of the less complex components.
Even though we had enough requirements to fill a phone book, I’d say nobody really knew what we were making. This is because the system was so complex that nobody could fully understand the downstream impacts of any change. There were times where the requirements simply stated “preserve existing functionality” because the BSA just didn’t know what that was.
There was a stigma of being in QA at the company I was at; being in QA was a thankless job. I remember a concerned BSA had asked me if I was forced to join the QA team, as a sort of demotion. When I replied that it was what I wanted, she looked like she was thinking “you poor thing, you actually believe that”.
What was interesting is that my testers had a lot of knowledge about the system and some of them were very highly respected. What worked really well was where my QA and the COBOL dev silos were bridged and they worked well together. The other 3 QA teams of mine had no such luck and so even with the same tools, we couldn’t improve as much.
In an ideal world, my testers wouldn’t be running most of the tests. They’d write the tests and manage the test data or even do exploratory testing to develop a better understanding of the system. I found that most developers just aren’t interested in doing that work but it needs to be done. The best person to run the test is the developer who’s going to fix the code. The more other people get involved, the longer the feedback loop and the more expensive it becomes to fix the code.
Most folks in my QA team didn’t come from a technical background. When one of my testers left the job shortly after I joined the team, it was suggested to me by development director that I could find good testers at the call centre. This is because there was this view that to be a tester, you need to know how to use the application and the call centre folks knew how. Interestingly enough, to be a developer you needed no such knowledge and so jobs could be outsourced …
Typically, folks try to automate a manual test which is the least useful because e2e is really slow. Also as he says, there’s a lot of implicit asserting. Not just the asserting but even data setup. When I first suggested to my testers that the developers would run the same automation they’re running, they had their doubts about it finding defects. This is because they weren’t putting in all the data needed for the setup or assertions. In fact, some tests were written like mental notes; a developer just couldn’t tell what was being tested specifically. I aimed to test the most complex component automatically because it created the most difficult and time-consuming defects.
I started with coarse grained unit tests. I actually at the time thought of them as component tests. Later I saw that the developers were testing the system the same way as my QA team. I wasn’t sure whether to say the developers were doing component tests or my team was was doing unit tests. Eventually I came across Ian Coopers presentation and I’d now say they were unit tests. As for tools at the level of whatever you’re writing, this is where being able to transform our DSL into whatever COBOL code/files were needed was helpful. Even though we were using Robot Framework or Cucumber, we didn’t need the COBOL developers to know or use Java just to run a test.
The biggest barrier for me was not the technology, it was the culture. In the COBOL dev team, the developers and testers had a great relationship long before I got there. Sadly in the other teams, they and their managers thought it was someone else’s job. A director literally told me to shut-up and go do my job over a really low severity defect that I wanted the dev team to unit test.
Trying to retro-fit the code base to make automated testing work is constrained by the architecture. The system was developed without the constraint of being testable except by end-to-end testing and so they were untestable. One example was that in production, customers couldn’t delete a record, they could just terminate it and create a new one with a different effective date. So the architects prevented any deletion operations being implemented which would make it hard to re-run tests in QA without creating more new data.
Because of such constraints, we started at a high level or component/coarse-grained unit tests. This meant we needed the entire engine and database spun up just to run one test. We’d then work on the edge of this reduced system and inject data via its file upload processes.
At the time I didn’t know what approval testing was but I think my testers did something like it. When they worked with developers in the lowest environment, they’d make a test, and then run it. Though the test had some assertions, they’d see if what the code did match it and if it didn’t, they would have updated the assertion. The reason for this is because some of the math was really complex and depending on where the rounding would take place in the system you’d get false failures.
This made me think of test automation performance. When I joined the QA team, the COBOL testers used to run test automation for new features only at the end of the manual testing cycle. Their automation would submit all the claims and then extract all the response from a database table. When I suggested running each test and its assertions, folks hesitated thinking it would take them longer and slow them down. The reality is that it did, it was twice as slow to be honest; instead of 5, it would take 10 minutes etc. The trade-off was that it could be run more frequently with the possibility of one test being executed at a time.
It’s a prerequisite that devs want to run the tests. They don’t need to even automate it, my enabling/platform team would help with that. I waited for a developer that really wanted to and he became the tip of the spear. Once he felt good and shared his positive experiences, only then did I approach others. Whatever tools we used had to make it easy. So if they were using a CPHA3 simulator that was homegrown, I started with that. I didn’t try to sell them on SoapUI or Cucumber etc, that came later.
My test automation developers eventually stopped automating tests. They worked on the platform that let developers and testers auto-generate tests in whatever tools they were using. They also worked on coaching and upgrading the manual testers. We made it less work for the developer to run the test than not run the tests (find and fix defects). In fact as more time went by, and I learnt how the developers worked, I had a bias to make it easier to run the tests for them instead of my own testers.
I mentioned that the testers used Cucumber (with Java) but the developers were COBOL developers. So how does that work out? Basically the .feature files were converted into a UML model which then was converted into whatever we wanted. When Dave talks about the framework constraining you, I realised that there was no version of Cucumber for COBOL or Cucumber for PLSQL or Cucumber for webMethods etc. Just because my team liked writing tests in Cucumber doesn’t mean it would work for the developers. In order to decouple the specification from test automation implementation, I had hoped my team would just move to AsciiDoctor or Markdown. That would then give us the flexibility to support any language via a library or REST API that could query the test model for the data.
Adoption is a journey, you can’t just tell someone to go do the thing. I agree with Henry that you should first have a bias for adoption and then for scaling because adoption is the hard part. I couldn’t tell testers to just switch to Cucumber. I mean I could but I assumed back then and still do think that it would have either been stressful or they wouldn’t have stuck with it. This is why Start with Why was so useful, I needed to focus on the early adopters before I focused on the majority. Just as he did, I noticed that once the developers could run our QA tests (which were basically unit tests) they now had more time and were more willing to create their own more low level tests. Their tests would now focus on failures rather than meeting the requirements.
Meeting the code base where it is reminds me of the Neo Nurture incubator. We met the code base where it is by generating data or automation in a format that worked for the developers. These were files uploaded by their COBOL scripts or submitting claims via the homegrown claim submission tool. Later they moved to SoapUI and then Cucumber.
How to write units tests without letting the whole system run? Well you let the whole system run as described above. One of the constraints of this approach is that we’d have leaky tests if we ran them concurrently. Everything was run sequentially as a result which could take several hours. Thankfully most legacy suites ran in a few minutes and by the time we got to Cucumber we were running individual tests in fractions of a second. We also did functional isolation, the QA team was already doing this.
I’m going to start with stating that if it wasn’t for a DSL and Eclipse Xtext, my entire transformation would have failed. You start with why but you also need a how. The reason we were able to move as fast is because of auto-generation of the test automation. This meant that test automation developers were no longer automating any tests, they were out of a job. They’re new job was to automate the creation of test-automation. This is no different that someone manually assembling a car was a robotic assembly line.
Let’s say in the case of the COBOL dev and testers, there was a third person, the test automation developer. How would that work out? Everytime the dev and tester had a conversation, you’d need this third person like a third person on a date. Each time you finish having a conversation, you’d need to make sure this person is also on the same page. Each time they make the code, there’s a chance it’s automating the wrong thing. Each time they make the code, there’s a delay, a non-value added activity in the lean world. Instead by automatically creating automation from a formal DSL, from the moment an tester and dev have a conversation, there could be automated tests on the developers laptop in less than a minute.
What Henry describes about no-one writing tests reminds me of how the UAT team approached it and so it’s interesting how they had the same outcome. I think it’s because both treated these tools as test automation authoring tools rather than a language for communicating with the business. Also interesting about my QA team’s test case writing style was that it was so clear that it could be given to our customers for review. It was truly living documentation in my biased eyes :). I should note that we didn’t use Gherkin even though we used Cucumber. Instead of given, when, then, we just used the asterisk for every step.
When we were using Robot Framework, for various reasons, the DSL grammar wasn’t very strict. That meant that we’d wind up with keywords like:
Later, I could use regular expressions to find similar keywords and remove duplicates. Eventually we adopted the Xtext framework with prevented these mistakes from happening.
I too only use DSL for higher levels of testing. My current GitHub projects that demonstrate what I did on the QA team use the DSL editor to write tests and generate test automation. The problem is when you want really low level tests, it’s no longer a trans-literation exercise and so gets very challenging. You can attempt it but then the code that creates the automation becomes really hard to understand which adds its own complexity and therefore sources of bugs. Admittedly I don’t have as many plain solitary unit tests as I’d like.
I mentioned earlier that I preferred we left Cucumber. One thing I had experimented with is letting the team write .feature files and I extended the language. That is, in our xtext editor allowed for a .feature file to have a Step-Object keyword followed by Step Definition, Step Parameter etc. I was continually trying to make writing the test case as easy as possible. Specifically I was trying to make writing a test case from which test automation could be derived as easy as possible. I was also trying to make it easier for a non-SME to write a test case, kind of how I as a developer can use APIs to make something. Also when a Cucumber test would fail, the stack trace would be really hard for someone to go through. Being able to have our own plain junit/TestNG automated tests would have reduced that toil on the team.
I didn’t think of it this way until Henry mentioned it but since we made the framework we could change it. My team organically evolved through the tools, almost one per year until we got to Cucumber. Every change was the result of the PDCA and Kaizen. Had I just forced the adoption of some testing framework, I don’t think we’d have been as successful because folks wouldn’t suggest improvements thinking they have to use it as is.
I think the developers should maintain the test automation. There are tools like cucumber that have both specification and implementation. They don’t give you an API or library to browse through the specification and make your own implementation. This makes it harder to adopt and organically merge into an existing way of working in the dev teams. This is where Xtext helps because it creates that API that lets you do this. I frequently told my dev team that paying someone else to automate a test or even run it was like delegating someone else to go to the gym for you. They’ll use the tools, have the community and know what to eat to get the awesome bod but it should be you doing those things.
I described in the Deming Driven Testing site that the dev teams were users of the data/feedback platform my QA team was trying to create. In order to create empathy for the dev side of things, I slowly tried to get the testers to work like developers. An example was having them understand the value of earlier feedback and how without it, there’s more mistakes and rework like they experienced with writing tests. I had the same QA team as their long time manager yet things were markedly better when I managed the team. The difference was that it wasn’t the people but the system within which they work. I was simply applying my novice level understanding of Dr Deming’s 14 points and SoPK.
I took a lean or continuous improvement approach for reasons I explained here. I described this process in this blog post, basically I never needed a budget because of the small steps are removing rather than adding. The goal was to reduce the waste after identifying the overburden that caused the fluctuation or in lean terms, reduce muda after identifying muri that caused the mura. The metrics that showed things were getting better were QA execution time or defect remediation time or test automation creation time; everything went down. Test automation coverage went up and so did developer and tester utilization. I also agree that the moral argument isn’t the way to go. My own team would hesitate at dropping test cases thinking it was the wrong thing to do. I had no problem with doing so because I knew that quality can’t be inspected in. Dr Deming showed the Japanese how quality can be used for their competitive advantage, it wasn’t about being morally right.
So if everything was going as well, why did I leave my job. Well, it was all going well until it didn’t. At some point the managers of the COBOL developers didn’t like the idea of their devs running the tests. They even tried to get them to stop it. I think it’s because folks weren’t comfortable with the idea that the QA team wasn’t the last to touch the code. This place was pathological, if there were problems in production, QA was asked why didn’t you test that. With QA no longer running the tests, and the developers taking a bigger role, that wasn’t going to fly. I also mentioned that my boss’s boss didn’t like me much and neither did my boss’s peers I think the usual push back of there’s no money for this didn’t apply since we didn’t need it. The other push back of are we doing development wrong also didn’t work because the developers willingly and enthusiastically adopted it. Everyone just wanted things to stay the same; all the lean stuff was really foreign to the culture.
]]>I’ve summarised the relevant parts of the talk into these sections.
To Toyota this is the order of importance:
The way it’s seen is overburden causes unevenness which in turn causes waste. Eliminating waste needs unevenness addressed which needs overburden addressed.
Waste is easy to see, you only need to observe a process once. Fluctuations (unevenness) are harder, you need to observe the process at least once. The people who know the fluctuations best are the ones on the shop-floor.
NOTE:
Overburden was caused by having too many specialists and not enough generalists or full-stack testers. This caused fluctuations in people’s utilization in the form of the famine/feast cycles that sometimes appeared. I mapped the 8 wastes to the QA inspection process here:
Studying the fluctuations was where SPC came in; I wanted the process of inspection under statistical control. I wanted to reduce the occurrence of last minute high priority defects which have very little time to be fixed by devs and re-tested by my team.
The first goal is to stabilize production by reducing fluctuations. Questions to ask are:
Sometimes there’s so much fire-fighting. It’s like you can’t stop to sharpen the axe because you have so many trees to fell. If you’re in a burning house, having a fire-fighter is nice but even better is to not have a burning house. Rewarding only the fire-fighters can create career arsonists. Reward the fire-safety inspector as well.
Overburden is a lack of safety; folks get hurt and then they slow down. First identify where are things going wrong, you’ll need some sort of data Once you have data, do this
Even for someone on the shop-floor, it’s hard to watch and do a task at the same time; you’re too close to the problem to see it. You can actually notice but it doesn’t mean you can make the connection about what’s causing it. Management needs to get rid of the cause.
Note:
The tree analogy reminds me of the cart with square wheels comic .Questions I asked:
- Is the test automation stable? Are the environments?
- Are there work-standards for writing tests? Are they effective, can someone execute tests written by someone else? Can you pair up basically.
How to free-up the time of an overburdened tester who’s busy with their own work or helping put out fires? One way is by adhering to standards.
If I wanted those who were free to be able to swarm to reduce the load on one tester, there had to be a test writing standard that would allow this. Other times, a non-SME would use Robot-Framework to automate the tasks for a SME. This is where the ubiquitous language came in; it was to standardize work and then reap the benefits.Finding an overburdened tester is easy, they’ll be falling behind schedule. The overburden caused by lack of safety to me was psychological safety.
There was this culture that the good testers find bugs and so they’d keep increasing the scope of testing till they found some afraid that not finding any reflected poorly on them. Only management (I) could change that.
You can’t get rid of fluctuation but it can be reduced with PDCA. In Western companies, it’s little planning, lots of doing, then even less checking and acting (sharing).
At Toyota, there’s huge emphasis on plan, way more than half. They try to understand what’s happening, with who, when. You need to stratify the problem, make a prioritised problem. Multiple factors might contribute to quality problems but pick the one that has the biggest impact. Now that you know the problem, what’s the root cause? Is it a process, worn-out tool, a setting? Sometimes you use the fishbone diagram or a mind-map. Develop multiple root-causes. In the Western world, the minute you find a root cause, there’s a rush to solve that. In Japan, for each root cause you ask this about the solution:
Then finally you get to the D in PDCA. Then check if it’s gone or only gone when no manager is looking. If it’s gone, then share the fix with others, that’s the act part. Sharing at Toyota is done by uploading their A3 to a common database so that their employees can Google that. If it’s not gone, restart the PDCA.
Note:
Stabilizing the fluctuations by standardizing work to me was different from reducing them through PDCA.
Standardizing didn’t need me to change anything like the PDCA cycles did.
The former was easier than the latter because introducing something new has more resistance to it.To me the first goal was to reduce common cause variation through PDCA cycles. Each week I met with everyone on my team for 1:1 to ask what’s bugging them so we can fix it.
Together we’d come up with an Kaizen which was either a Jidoka or a Poka-Yoke.
How to change from push to pull. Pull is a system to manage inventory for made to stock or manage workload if it’s made to order. Inventory is a necessary evil used for short term decoupling of fluctuations. You need at least one part to produce and more than one to buffer.
Kanban is a inventory management system. You set your kanban system to represent your target inventory. Let’s say you want to have 500 hinges, it should cover the time to replenish the inventory and reduce the fluctuations.
Kanban is typically understood as a two bin system which is a system with two Kanbans (information) but can be more than 2.
How many bins you need depends on the replenishment time, how long to get the bin back. You need 1 bin per day, if you can fill a bin in half a day, two bins are enough. If it takes 2 days to fill the bin, then after two days, you have a day without parts so you need 3 and for safety maybe a 4th bin. The equation is replenishment time divided by takt time but don’t take the average, take the extreme to be safe. Ideally you want to have a long replenishment time divided by short takt time but you don’t want to have millions of hinges. At some point you will run out and then it’s OK to start fire-fighting with rush orders etc.
Note:
This is about using kanban bins (git commits) to manage made to stock.
What’s the stock here? Instead of orders or hinges, it was unexecuted test cases/stories/scenarios.
The goal is to keep the number commits on branches in git close to 1. As explained in the just in time section, the testers aimed to produce enough test cases for a developer so they could code for at least a day.
They had to make enough test cases for the day and even the next day in case something came up and they couldn’t work that day etc. No inventory is the goal but with the dev managers and I having different priorities, we needed this buffer as a mitigation. If the developer was moving fast, the testers would do overtime and relax the next day.
IKEA is an exception by making to stock and then you mix and match but kitchens are usually made to order. For made to stock, you want high stock availability with least material to produce it. For made to order, you don’t have the parts and the customer always has to wait. How fast to make it aka lead time is influenced by utilization. If you want a shorter lead time, you need higher utilization, but then parts must always be available.
For a made to order system, you use CONWIP; Constant Work in Progress. It’s like a Kanban card but you attach a paper to it with the kitchen spec (I’d say it’s like an envelope). When the kitchen is made, the paper (letter) is archived and the card (envelope) goes back to the start. You limit the number of kitchens being worked on by cards where a card represents a bay to make a kitchen.
Assume you have limited capacity such as a person who is needed for 10 kitchen projects. They need time to understand what’s going on so all the time spent on the project is just catching up. Perhaps they’ll do bare minimum for 8 projects and only work on 2. Statistically 3 is a good number per person to balance the work load.
If people are spread too thin, efficiency goes down. Too many projects creates overburden, then you have fluctuations (unevenness) and lead times increase. Pull systems for made to order limits the number of kitchen projects worked on at any time. For that system to work, we need a pool, a funnel of work waiting so when the card comes back, you can start the next project. If there’s no orders, you have the problem anyways of people idling, pull system or not.
Note:
What he describes here to me is the Kanban board that everyone uses where CONWIP is a WIP limit.
I see my QA team working on features as a made to order process because like a custom kitchen, no two were the same. That said, some of the internal processes were treated as made to stock with git as the inventory management system.I tried to set strict WIP limits, basically one feature per person and at most 1 production issue at any time.
However we spent quite a bit of time waiting for information on the prod issues so then we had to increase those WIP limits.
You have small and big kitchen orders for eg and you order parts, sometimes too many at once. This was seen with cars during COVID; there were no orders and then there was an avalanche. The suppliers weren’t happy because they had all the parts but no orders and then they had orders but no parts and had the costs of expediting the ramp up.
A pull system doesn’t improve or prevent fluctuation, but prevents it from becoming worse because the CONWIP acting as a limiter reduces the effect. By limiting the number of projects in progress, you make it more predictable to see how long it takes to do the project. If the project quantity fluctuates from 5 to 50, your lead time also fluctuates and that makes it harder to predict.
The pull system is a good platform to make systems on top of them like leveling systems. Let’s say you’re making 2 door and 4 door cars. If you do all 2 doors and then all 4 doors, you’ll have bored folks and then overworked folks. Instead alternate 2 door and 4 doors, this is levelling.
Levelling is tricky, first stabilise system (standards, good maintenance), then have a pull system to reduce fluctuations and then think about levelling. You want to first get your car running reliably before tweaking the performance or fuel-efficiency. Different ways to do levelling.
Make the same number everyday. 275 crankshafts every day, more than 10% change from month to month, this is a daily production plan and it’s difficult. The Germans have a monthly pattern rather than alternating. The problem is you stick to a plan for the whole month which is hard since most companies can’t do today what they planned yesterday.
Easier is to mix them to the smallest possible lot size, ideally 1 which isn’t always possible. Don’t do all 2 doors this week and all 4 door next week, you keep both sub-assemblies equally busy by intermingling them. This is constrained by the time it takes to switch over from one way to make them to another. The more often you switch tool, the lower the inventory, the lower the work for all systems around it.
Levelling needs high reliability and stability aka low fluctuation of your system. Capacity, inventory and time are the ways to decouple fluctuation. Toyota can do it because they control the time when you’re getting your car. The dirty secret is that that you would increase your lead-time just to do more levelling. It’s like intentionally slowing down the flow rate by buffering for a smoother experience. This only works if you have a pattern you can stick with.
]]>Note:
When I think of levelling, there’s two types, levelling with new features and production bugs.
The goal is to not have accumulation of untested code commits in git. Put another way, for two features of the same size, you don’t want to have two vastly different lead times as a result of working on them in sequence.