I generally agree with the scenario you’ve set up but there are some misunderstandings here.
This is a total of 2N + M + 1 messages for a single job, which can be serious processing overhead depending on the nature of the task.
This is a measure of difficulty not complexity. You’re right that with a large network size this makes a lot of data that is more difficult to process, but it’s actually simple not complex. The complexity of this is still fairly simple a linear O(n) because your messages grow linearly relative to the amount of jobs your process.
A more complex scenario is that if your design had a flaw that generated 2N^2 + M + 1 messages by adding a superfluous for loop somewhere, then your complexity would be O(n^2) relative to the amount of your jobs your process.
If you’re digging a hole difficulty is based on how mechanized/big your shovel is relative to the size of the hole. If you’re digging a 2x2 hole 6 feet down with a gardening trowel it’s hard, it’s easier with a full size shovel, it’s incredibly easy with a backhoe.
If you’re digging a hole, complexity on the other hand is what you must do to prevent all the ways that the earth can collapse on you based on the dimensions of your hole, the type of soil you dig in, etc. A 2x2 6 foot hole straight down is a less complex than digging a 2x2 hole 6 feet horizontally into a mountain, because your hole can collapse not only based on the sides but from the cave-in above. Despite removing the same volume of earth you have a more complex task because you must scaffold to remove the risk of cave-in. There’s also dimensions of soil composition etc.
Complexity typically affects difficulty but difficulty never affects complexity. Sometimes complexity can also be fully detached from difficulty it depends on the actual problem.
You are making the same mistake OP made.
The controller sends a job to some worker. The controller knows which workers have spare capacity because it has memory of how many jobs it gave to which worker and how many jobs have been returned.
This is where you hide the actual complexity in controller based designs. The complexity of offloading the state management from the individual node to a controller node is not simple in the general case. Hierarchical organization here only reduces the difficulty of processing by reducing the amount of data overhead. Reducing complexity here comes through standardization of nodes in the system not through hierarchical organization.
You can make really simple assumptions about capacity if your network is made of nodes that are all the same (and your workload domain is well bounded and understood), however as soon as you vary the specifications of the nodes and the workload requirements this becomes an incredibly complex calculation to figure out where a job can go based on available memory, compute, storage and other needs.
This becomes even more complex if you need to handle specialized functions that exist only on certain nodes and not others, such as availability of NPUs for AI tasks, availability of hardware enabled transcoding for media management tasks, availability of randomization hardware for crytpographic tasks, etc.
Ultimately this is why horizontal job runners based on messaging middle-ware patterns (0mq, rabbitmq, Kafka) still exist and are used in highly complex environments.
Moving on to the social implications of this approach, reducing complexity this way is quite literally not applicable to human systems. You cannot ensure fungible nodes for humans where each node has the human equivalent of 32 GB RAM, 4 VCPUs at 3.2 GHZ and 100 GB storage. If we did that to humans that would be called eugenics.
You cannot make the complexity from non-uniformity in human systems superfluous like you can in computer systems. It’s immoral. Humans are not computers.
I can program a system to work any way that I want sure. If I want local durability I can make a mesh. If I want to lower the communications overhead I can centralize decision making in a controller structure.
I cannot program humans in the same way and nor should I think about it as such because:
Humans are not computational machines, they have their own motivations, biases and needs
Programming humans this way is not just impossible but at best is just paternalism by another name warts and all.
To apply system design to humans a system has conceive at minimum of the dimensions of power and faith to be durable against:
Humans with powerful roles in the system who are acting in bad faith causing harm for personal benefit
Humans with weak roles in the system that are acting in good faith against a resilient system that is effectively harming them
Humans with powerful roles in the system who are acting in good faith but are making mistakes that lead to harm that they do not recognize/dismiss
Humans with weak roles in the system that are acting in bad faith against the system
Humans with powerful roles in the system that are acting in bad faith against the system.
This is a level of complexity that is much harder than designing a job runner with specific properties for your use-case.
Socialist theory-crafting in 2025 at the level of typical systems design maps is not helpful or insightful. It’s the equivalent of physics research in 2025 using the Bohr model. In tech terms, your base of concern should start at things like durable consensus algorithms and Byzantine faults. If you start there you are ensuring that the technology you apply to your social organization explicitly protects in priority order the values or outcomes you specify. I think this is a viable method for human organization especially because it encodes the typical outcomes and injustices that are part of the system’s function explicitly. Everything is a trade off.
However concerning yourself with communications overhead is a 20th century problem. We have technological solutions that minimize those concerns, we simply haven’t developed political systems matching our level of technology. Building durable cybernetic systems like Project Cybersyn or OGAS are the pathways to ensuring a systemic socialism when these systems are open, transparent to average citizens and developed in partnership with their needs.
TL;DR, my criticism here is that we are talking about aspects of systems design that are only important historically and the conversations focus ultimately misleads us in how we build these systems, leading to worse outcomes. We are mainly talking about the difficulty of human communication between horizontal and vertical structure, but not about the complexity of human relations those systems apply to.
The complexity of this is still linear O(n) because your messages grow linearly relative to the size of your network.
In terms of big O complexity you are correct that it is linear growth. However, the other solution has constant messaging complexity. So there was still an improvement in complexity.
Although, I should also clarify that we were not exactly discussing big O complexity to begin with, and the use of the word “complexity” was a bit more colloquial.
The complexity of offloading the state management from the individual node to a controller node is not simple in the general case.
This is true. I should have been more clear about this. Most likely with more complex and less standardized actors there will need to be periodic re-synchronization with the controller since the controller will only be estimating the state of the workers based on memory (which will become more inaccurate over time).
On a per job basis, there is still more or less the constant overhead instead of linear overhead. It is just that overall for the organization there will be linear overhead. However, in the purely horizontal case, the organization overall will have a quadratic overhead (since the total number of links is O(n^2))
Hierarchical organization here only reduces the difficulty of processing by reducing the amount of data overhead. Reducing complexity here comes through standardization of nodes in the system not through hierarchical organization.
On the contrary, I think it is the other way around. Standardized nodes need to share less information to each other because the nodes can make more assumptions about other nodes. This reduces data overhead. However, the architecture of the network itself determines the node complexity.
Ultimately this is why horizontal job runners based on messaging middle-ware patterns (0mq, rabbitmq, Kafka) still exist and are used in highly complex environments.
I cannot comment on this, as my knowledge of computer science comes largely from random electives I took for fun.
You cannot ensure fungible nodes for humans
This is a given, however, nowhere in my example was the internal nature of the nodes themselves assumed to be uniform. The messaging schemes I have described, for instance, could be used with actual humans, with some modifications, namely that some of the “messages” would likely be more like “meetings”, and the workers would have to periodically meet with the project planner and discuss things.
I cannot program humans
I am not talking about programming humans, but the schemes they use to coordinate with each other. Those are 2 entirely different things.
To apply system design to humans a system has conceive at minimum of the dimensions of power and faith to be durable against:
Democratic centralist systems always stress accountability for people in powerful positions. This is the reason why ML states often get slandered as being “corrupt”, because in ML countries, such corruption is constantly being exposed by design whereas in capitalist countries, corruption is simply the norm and goes unnoticed.
A horizontal system could also harm you. The free market is a relatively horizontal system, and it keeps harming us. The very decentralized nature of the free market makes it so that you often have no recourse or no one to hold accountable when things go wrong, because the actions of the entire group/ dynamic of the system is the problem.
This is a level of complexity that is much harder than designing a job runner with specific properties for your use-case.
The example of the job-runner is not meant to be one that you can build a society on. It is simply the comparison of some of the simplest possible schemes for communication for a project. It exists to illustrate the complexity reduction that can be achieved with a hierarchical design for a specific problem class.
It’s the equivalent of physics research in 2025 using the Bohr model.
But this is not research, this is simply an internet forum. I appreciate the things you are telling me, but I did not come here knowing these things. I am also not saying that the communications overhead problem is a problem that needs to be solved on the theoretical level.
Building durable cybernetic systems like Project Cybersyn or OGAS are the pathways to ensuring a systemic socialism when these systems are open, transparent to average citizens and developed in partnership with their needs.
Although, I should also clarify that we were not exactly discussing big O complexity to begin with, and the use of the word “complexity” was a bit more colloquial.
Which is why my original post offered the term efficiency that OP rejected and insisted on using complexity.
I am not talking about programming humans, but the schemes they use to coordinate with each other. Those are 2 entirely different things.
These are not “simply” coordination schemes since they are created entirely to administer the distribution of resources in the real world. They are social meta programming. You are arguing beyond aspects of coordination/communication you are arguing about real world stakes.
This is a given, however, nowhere in my example was the internal nature of the nodes themselves assumed to be uniform. The messaging schemes I have described, for instance, could be used with actual humans, with some modifications, namely that some of the “messages” would likely be more like “meetings”, and the workers would have to periodically meet with the project planner and discuss things.
Again you are abstracting away the purpose of such a system in order to prove its specifications in a vacuum rather than using the actual intent. What are the meetings about? Production? Great we’re back at node fungibility due to distribution of labor and individual human ability.
Democratic centralist systems always stress accountability for people in powerful positions. This is the reason why ML states often get slandered as being “corrupt”, because in ML countries, such corruption is constantly being exposed by design whereas in capitalist countries, corruption is simply the norm and goes unnoticed.
Sure. I agree. However:
Your argument here is based on a hypothetical demcen system full of good faith actors.
The reason that in capitalist countries corruption becomes legalized and normalized is because “corruption” in the general case boils down to perception. We’re back at interpreting good faith vs bad faith. Capitalist propaganda works overtime to manage the perception of the legalized corruption in capitalist countries to at best normal expressions of rights and at worst good faith mistakes.
A horizontal system could also harm you. The free market is a relatively horizontal system, and it keeps harming us. The very decentralized nature of the free market makes it so that you often have no recourse or no one to hold accountable when things go wrong, because the actions of the entire group/ dynamic of the system is the problem.
It’s entirely arguable if the market is in practice horizontal or hierarchical. Regardless my arguments here aren’t about proving horrizontalism as “better than”, it’s about the complex trade offs happen socially when choosing these broad structural archetypes to organize around. I never said horrizontal systems cannot be harmful. My point is that the difficulty of horizontal systems can be a form protection.
But this is not research, this is simply an internet forum. I appreciate the things you are telling me, but I did not come here knowing these things. I am also not saying that the communications overhead problem is a problem that needs to be solved on the theoretical level.
My point is based on OPs original submission and my understanding that your position was a defense of it. OP submitted OP’s own essay from OP’s own substack with a pro-heirarchy case. I criticized the essay as generally lacking depth on the subject. OP got mad at me and started berating me based on OP’s socialist bona fides.
I could have been gentler with my criticism because I did not understand that OP posted OP’s own substack because the names don’t match. I understand that not everybody knows these things, my line of criticism while admittedly harsh and teetering on unproductive, is honestly meant to raise the conversation from simplistic defenses of hierarchy using typical bromides of the immortal science to a more holistic evaluation of these systems and their implementation and conception of socialism.
Your argument here is based on a hypothetical demcen system full of good faith actors.
Part of the point of the DemCen system is to increase the prevalence of good-faith actors and decrease the prevalence of bad faith ones. It is not as if MLs have historically just ignored education of the youth or the cultivation of good faith actors.
Regardless my arguments here aren’t about proving horizontalism as “better than”
Part of the point of the DemCen system is to increase the prevalence of good-faith actors and decrease the prevalence of bad faith ones. It is not as if MLs have historically just ignored education of the youth or the cultivation of good faith actors.
This doesn’t practically explain the how. You’re talking about building patronage networks. That effectively means that those networks themselves can be evaluated as good or bad faith. This just moves the question to another area of the political structure typical of DemCen systems.
Then what are we even arguing about?
When reading theory I don’t see us as being on “teams”. What’s the purpose of writing this stuff if we’re evaluating this solely through the lens of “teams”? At that point any article might as well say “Right Marxist Leninism is the best”. It’s a bad argument it could be better. I’m not even settled on what the “ultimate system” here is, I don’t think there is one, especially not as simple as just do “hierarchy”. If we want to converse about this stuff in good faith we should generally have a complex conversation about it in reality, that often means there’s a lot of grey areas and not a lot of practically right answers if we’re being honest with ourselves.
Like I’ve said elsewhere in this thread: CPC’s done a really good job, but they’re not in the business of exporting socialist thought. I linked this
The CPC’s own words are:
Second, while continuing to learn from and absorb ideas from other countries, we have made our own explorations and innovations to avoid blindly copying the system or democratic model of any other country
While leading the people through the periods of revolution, construction, and reform, the CPC always held fast to the belief that no two political systems in the world are the same and that no universal political model exists.
This is a very adult way of speaking about socialism compared to other sets of thinkers who argue implicitly about universal contexts, and universalize ML concepts without explaining their application to their context or even explaining what their context is. If you believe in whatever theory and whatever individual practices, the onus is on you to explain why those practices work in the contexts you’re referring to. OP has not done this vis-a-vis hierarchy. They have created a very abstract rationale.
I generally agree with the scenario you’ve set up but there are some misunderstandings here.
This is a measure of difficulty not complexity. You’re right that with a large network size this makes a lot of data that is more difficult to process, but it’s actually simple not complex. The complexity of this is still fairly simple a linear O(n) because your messages grow linearly relative to the amount of jobs your process.
A more complex scenario is that if your design had a flaw that generated 2N^2 + M + 1 messages by adding a superfluous for loop somewhere, then your complexity would be O(n^2) relative to the amount of your jobs your process.
If you’re digging a hole difficulty is based on how mechanized/big your shovel is relative to the size of the hole. If you’re digging a 2x2 hole 6 feet down with a gardening trowel it’s hard, it’s easier with a full size shovel, it’s incredibly easy with a backhoe.
If you’re digging a hole, complexity on the other hand is what you must do to prevent all the ways that the earth can collapse on you based on the dimensions of your hole, the type of soil you dig in, etc. A 2x2 6 foot hole straight down is a less complex than digging a 2x2 hole 6 feet horizontally into a mountain, because your hole can collapse not only based on the sides but from the cave-in above. Despite removing the same volume of earth you have a more complex task because you must scaffold to remove the risk of cave-in. There’s also dimensions of soil composition etc.
Complexity typically affects difficulty but difficulty never affects complexity. Sometimes complexity can also be fully detached from difficulty it depends on the actual problem.
You are making the same mistake OP made.
This is where you hide the actual complexity in controller based designs. The complexity of offloading the state management from the individual node to a controller node is not simple in the general case. Hierarchical organization here only reduces the difficulty of processing by reducing the amount of data overhead. Reducing complexity here comes through standardization of nodes in the system not through hierarchical organization.
You can make really simple assumptions about capacity if your network is made of nodes that are all the same (and your workload domain is well bounded and understood), however as soon as you vary the specifications of the nodes and the workload requirements this becomes an incredibly complex calculation to figure out where a job can go based on available memory, compute, storage and other needs.
This becomes even more complex if you need to handle specialized functions that exist only on certain nodes and not others, such as availability of NPUs for AI tasks, availability of hardware enabled transcoding for media management tasks, availability of randomization hardware for crytpographic tasks, etc.
Ultimately this is why horizontal job runners based on messaging middle-ware patterns (0mq, rabbitmq, Kafka) still exist and are used in highly complex environments.
Moving on to the social implications of this approach, reducing complexity this way is quite literally not applicable to human systems. You cannot ensure fungible nodes for humans where each node has the human equivalent of 32 GB RAM, 4 VCPUs at 3.2 GHZ and 100 GB storage. If we did that to humans that would be called eugenics.
You cannot make the complexity from non-uniformity in human systems superfluous like you can in computer systems. It’s immoral. Humans are not computers.
I can program a system to work any way that I want sure. If I want local durability I can make a mesh. If I want to lower the communications overhead I can centralize decision making in a controller structure.
I cannot program humans in the same way and nor should I think about it as such because:
To apply system design to humans a system has conceive at minimum of the dimensions of power and faith to be durable against:
This is a level of complexity that is much harder than designing a job runner with specific properties for your use-case.
Socialist theory-crafting in 2025 at the level of typical systems design maps is not helpful or insightful. It’s the equivalent of physics research in 2025 using the Bohr model. In tech terms, your base of concern should start at things like durable consensus algorithms and Byzantine faults. If you start there you are ensuring that the technology you apply to your social organization explicitly protects in priority order the values or outcomes you specify. I think this is a viable method for human organization especially because it encodes the typical outcomes and injustices that are part of the system’s function explicitly. Everything is a trade off.
However concerning yourself with communications overhead is a 20th century problem. We have technological solutions that minimize those concerns, we simply haven’t developed political systems matching our level of technology. Building durable cybernetic systems like Project Cybersyn or OGAS are the pathways to ensuring a systemic socialism when these systems are open, transparent to average citizens and developed in partnership with their needs.
TL;DR, my criticism here is that we are talking about aspects of systems design that are only important historically and the conversations focus ultimately misleads us in how we build these systems, leading to worse outcomes. We are mainly talking about the difficulty of human communication between horizontal and vertical structure, but not about the complexity of human relations those systems apply to.
In terms of big O complexity you are correct that it is linear growth. However, the other solution has constant messaging complexity. So there was still an improvement in complexity.
Although, I should also clarify that we were not exactly discussing big O complexity to begin with, and the use of the word “complexity” was a bit more colloquial.
This is true. I should have been more clear about this. Most likely with more complex and less standardized actors there will need to be periodic re-synchronization with the controller since the controller will only be estimating the state of the workers based on memory (which will become more inaccurate over time).
On a per job basis, there is still more or less the constant overhead instead of linear overhead. It is just that overall for the organization there will be linear overhead. However, in the purely horizontal case, the organization overall will have a quadratic overhead (since the total number of links is O(n^2))
On the contrary, I think it is the other way around. Standardized nodes need to share less information to each other because the nodes can make more assumptions about other nodes. This reduces data overhead. However, the architecture of the network itself determines the node complexity.
I cannot comment on this, as my knowledge of computer science comes largely from random electives I took for fun.
This is a given, however, nowhere in my example was the internal nature of the nodes themselves assumed to be uniform. The messaging schemes I have described, for instance, could be used with actual humans, with some modifications, namely that some of the “messages” would likely be more like “meetings”, and the workers would have to periodically meet with the project planner and discuss things.
I am not talking about programming humans, but the schemes they use to coordinate with each other. Those are 2 entirely different things.
The example of the job-runner is not meant to be one that you can build a society on. It is simply the comparison of some of the simplest possible schemes for communication for a project. It exists to illustrate the complexity reduction that can be achieved with a hierarchical design for a specific problem class.
But this is not research, this is simply an internet forum. I appreciate the things you are telling me, but I did not come here knowing these things. I am also not saying that the communications overhead problem is a problem that needs to be solved on the theoretical level.
I doubt you will find many MLs opposed to this.
Which is why my original post offered the term efficiency that OP rejected and insisted on using complexity.
These are not “simply” coordination schemes since they are created entirely to administer the distribution of resources in the real world. They are social meta programming. You are arguing beyond aspects of coordination/communication you are arguing about real world stakes.
Again you are abstracting away the purpose of such a system in order to prove its specifications in a vacuum rather than using the actual intent. What are the meetings about? Production? Great we’re back at node fungibility due to distribution of labor and individual human ability.
Sure. I agree. However:
It’s entirely arguable if the market is in practice horizontal or hierarchical. Regardless my arguments here aren’t about proving horrizontalism as “better than”, it’s about the complex trade offs happen socially when choosing these broad structural archetypes to organize around. I never said horrizontal systems cannot be harmful. My point is that the difficulty of horizontal systems can be a form protection.
My point is based on OPs original submission and my understanding that your position was a defense of it. OP submitted OP’s own essay from OP’s own substack with a pro-heirarchy case. I criticized the essay as generally lacking depth on the subject. OP got mad at me and started berating me based on OP’s socialist bona fides.
I could have been gentler with my criticism because I did not understand that OP posted OP’s own substack because the names don’t match. I understand that not everybody knows these things, my line of criticism while admittedly harsh and teetering on unproductive, is honestly meant to raise the conversation from simplistic defenses of hierarchy using typical bromides of the immortal science to a more holistic evaluation of these systems and their implementation and conception of socialism.
Part of the point of the DemCen system is to increase the prevalence of good-faith actors and decrease the prevalence of bad faith ones. It is not as if MLs have historically just ignored education of the youth or the cultivation of good faith actors.
Then what are we even arguing about?
This doesn’t practically explain the how. You’re talking about building patronage networks. That effectively means that those networks themselves can be evaluated as good or bad faith. This just moves the question to another area of the political structure typical of DemCen systems.
When reading theory I don’t see us as being on “teams”. What’s the purpose of writing this stuff if we’re evaluating this solely through the lens of “teams”? At that point any article might as well say “Right Marxist Leninism is the best”. It’s a bad argument it could be better. I’m not even settled on what the “ultimate system” here is, I don’t think there is one, especially not as simple as just do “hierarchy”. If we want to converse about this stuff in good faith we should generally have a complex conversation about it in reality, that often means there’s a lot of grey areas and not a lot of practically right answers if we’re being honest with ourselves.
Like I’ve said elsewhere in this thread: CPC’s done a really good job, but they’re not in the business of exporting socialist thought. I linked this
The CPC’s own words are:
This is a very adult way of speaking about socialism compared to other sets of thinkers who argue implicitly about universal contexts, and universalize ML concepts without explaining their application to their context or even explaining what their context is. If you believe in whatever theory and whatever individual practices, the onus is on you to explain why those practices work in the contexts you’re referring to. OP has not done this vis-a-vis hierarchy. They have created a very abstract rationale.