Our Evaluation Methodology.

We systematically and transparently evaluate OpenGov referenda in the Polkadot and Kusama ecosystems.

Our goal:

To make complex votes understandable and to reach well-informed voting decisions.

Our Process:

1. Information Gathering:

We research all available data on the referendum.

2. Analysis in 5 Dimensions:

■Impact on the Ecosystem

■Governance Compatibility

■Cost-Benefit Ratio

■Transparency and Traceability

■Record and Credibility

3. Question Catalogue:

Depending on the origin group (e.g., Spending Origins), we use a specific catalogue of 19 questions.

Learn more about our criteria for each group:

4. Evaluation:

Each dimension is rated with a score from 0 to 10. Thresholds determine the outcome: AYE (> 6.5), ABSTAIN (3.5–6.5), NAY (< 3.5).

5. Voting Decision:

Based on the analysis results we make our decision according to our voting table.

Question Catalogue for the Analysis of Polkadot and Kusama Referenda.

For each origin group, there is a defined question catalogue:

GROUP 1: System Change and Management Origins

Origins: [0]Root, [1]WhitelistedCaller, [10]StakingAdmin, [12]LeaseAdmin, [13]FellowshipAdmin, [14]GeneralAdmin, [15]AuctionAdmin

■ Impact on the Ecosystem

Evaluation Target: Addressing the question of whether the proposal strategically and sustainably strengthens the network.

  1. Question: Does the proposal demonstrably contribute to the long-term security, scalability, or decentralization of the network?
    Evaluation Focus: Examines whether the proposal promotes core network goals such as security, performance, and decentralization.
  2. Question: Does the proposal specifically address existing vulnerabilities or bottlenecks in the Polkadot ecosystem?
    Evaluation Focus: Questions whether the proposal resolves specific current network issues and is thus strategically sound.
  3. Question: Does the proposal align with Polkadot’s strategic direction and roadmap to promote the network’s sustainable development?
    Evaluation Focus: Investigates whether the proposed change fits into Polkadot’s long-term plan and thereby makes a sustainable contribution.
  4. Question: Does the proposal bring broad value to key actors and areas of the ecosystem (e.g., validators, parachains, end users) rather than just a small interest group?
    Evaluation Focus: Assesses whether the proposal’s benefits extend to the entire network, making it strategically significant.

■Governance Compliance

Evaluation Target: Addressing the question of whether the proposal is appropriately contextualized.

  1. Question: Is the proposal clearly within the scope of responsibility of the chosen origin (e.g., Root for system-wide changes), or does it overstep governance competencies?
    Evaluation Focus: Checks whether the proposal’s content aligns with the responsibilities of the selected origin and complies with regulations.
  2. Question: Are there precedents or previous similar proposals that demonstrate this proposal is being processed correctly through this governance path?
    Evaluation Focus: Examines whether past experiences support the choice of process and origin for this proposal.
  3. Question: Is the governance process being used meaningfully with this proposal, without bypassing or unnecessarily burdening established procedures?
    Evaluation Focus: Questions whether the matter is being handled properly within the existing governance framework and whether there is no misuse of processes.

■Cost-Benefit Ratio

Evaluation Target: Addressing the question of how efficiently resources are used relative to the impact.

  1. Question: Are the potential risks or negative side effects of the proposed change proportionate to the expected benefits for the network?
    Evaluation Focus: Considers the balance between the risks associated with the proposal and the intended positive effect.
  2. Question: Is the required technical effort or additional complexity introduced by the proposal justified by the achievable impact?
    Evaluation Focus: Assesses whether development effort, technical changes, or increased system complexity are offset by sufficient added value for the network.
  3. Question: Have alternative solutions with lower resource requirements been considered to achieve the same goal, and why was this change chosen?
    Evaluation Focus: Checks whether more efficient or simpler approaches were available and whether the decision for this proposal is well-justified.
  4. Question: Does the proposal create long-term obligations or maintenance efforts, and are these sufficiently justified by the sustainable benefits?
    Evaluation Focus: Questions the ratio of ongoing costs (e.g., follow-up costs, maintenance) to the long-term benefits of the measure.

■Transparency and Traceability

Evaluation Target: Addressing the question of whether the proposal enables evidence-based tracking and evaluation.

  1. Question: Is it clearly communicated what specific systemic changes are to be made and what goal is being pursued?
    Evaluation Focus: Examines whether the proposal clearly explains what will be changed and why, so evaluators can fully understand the content.
  2. Question: Is there sufficient information, technical details, or testing available to technically validate the proposed change and verify its necessity?
    Evaluation Focus: Checks whether the proposal is supported by documentation (e.g., code change details, analyses, test results) to enable a solid evaluation.
  3. Question: Are there clear success criteria or metrics to evaluate the impact of the change later?
    Evaluation Focus: Looks at whether the proposal defines measurable criteria (e.g., performance metrics, security indicators) to objectively assess the measure’s success afterward.
  4. Question: Are the decision-making reasons and the change process transparently documented (e.g., through public discussions, minutes, or reports)?
    Evaluation Focus: Assesses the transparency of the decision-making process—whether relevant discussions are disclosed and responsibilities clarified, so the community can understand the decision.

■Track Record and Credibility

Evaluation Target: Addressing the question of whether the proposer(s) are credible and capable of meaningfully implementing the proposal.

  1. Question: Have the proposers or their team already made successful contributions or similarly complex changes in the Polkadot ecosystem?
    Evaluation Focus: Examines the proposers’ past achievements—whether they have demonstrated through previous projects, code contributions, or initiatives that they can meaningfully advance the network.
  2. Question: What comparable projects or network improvements have the proposers implemented in the past, and what does this say about their ability to execute this proposal?
    Evaluation Focus: Looks at specific past successes of the proposers, especially those similar to the current proposal, to infer their implementation competence.
  3. Question: Are there publicly documented references, community feedback, or other evidence supporting the proposers’ expertise and credibility in this area?
    Evaluation Focus: Checks whether the proposers are supported by reputation—e.g., through community trust, forum contributions, or recognized expertise.
  4. Question: Does the team have the necessary technical expertise and organizational strength to effectively implement this far-reaching change in line with community expectations?
    Evaluation Focus: Assesses whether the proposers are technically and organizationally capable of successfully implementing the proposal, and whether the community can have justified confidence in their abilities.

GROUP 2: Signaling Origin

Origins: [2]WishForChange

■Impact on the Ecosystem

Evaluation Target: Addressing the question of whether the proposal strategically and sustainably strengthens the network.

  1. Question: Does the proposed signal address a core strategic issue whose resolution would promote the sustainable strength of the Polkadot network?
    Evaluation Focus: Checks whether the referendum’s concern is relevant and strategically significant for the network’s long-term development.
  2. Question: Can this Wish For Change referendum provide significant momentum for ecosystem improvements that extend beyond the immediate voting period?
    Evaluation Focus: Examines whether the signal could lead to actual changes or initiatives that benefit the network long-term, beyond just the poll result.
  3. Question: Does the signal aim to address an existing challenge or urgent development area in the network that is critical for the ecosystem’s future?
    Evaluation Focus: Questions whether the referendum addresses an issue or need whose resolution would sustainably improve Polkadot’s resilience or competitiveness.
  4. Question: To what extent does this signal mobilize or unite the community to collaboratively work on a strategic change for the benefit of the ecosystem?
    Evaluation Focus: Assesses whether the issue is significant enough to garner broad support and move the community in a constructive direction, indicating strong impact.

■Governance Compliance

Evaluation Target: Addressing the question of whether the proposal is appropriately contextualized.

  1. Question: Does the subject of this Wish For Change referendum fall within the scope of Polkadot’s governance, or does it address areas outside its influence?
    Evaluation Focus: Checks whether the signal’s subject can actually be influenced by on-chain governance or if it concerns matters beyond token holders’ direct control.
  2. Question: Is the choice of the signaling origin (Wish For Change) appropriate for this concern, or would another governance tool (e.g., Treasury Proposal or binding referendum) have been more suitable?
    Evaluation Focus: Questions whether the proposed issue is correctly placed on the signaling path rather than as a directly actionable proposal—whether form and content align.
  3. Question: Is the Wish For Change mechanism being used appropriately here (as a non-binding sentiment gauge) without bypassing or distorting established decision-making processes?
    Evaluation Focus: Examines whether the referendum genuinely serves to gauge community sentiment and is not misused as a superficial substitute for a binding decision.

■Cost-Benefit Ratio

Evaluation Target: Addressing the question of how efficiently resources are used relative to the impact.

  1. Question: Is the expected knowledge gain from this sentiment gauge proportionate to the effort required for its execution and community participation?
    Evaluation Focus: Assesses whether the insights or signaling effect gained from the vote justify the time and participation invested.
  2. Question: Could comparable information or community sentiment have been obtained less expensively (e.g., through forum discussions), or does the added value clearly justify the on-chain referendum?
    Evaluation Focus: Checks the efficiency of the chosen path—whether a formal on-chain vote was truly necessary to gauge this sentiment, or if simpler, less resource-intensive alternatives existed.
  3. Question: Is the governance system being used efficiently through this Wish For Change referendum, or does it represent an unnecessary burden without corresponding value?
    Evaluation Focus: Critically questions whether this referendum delivers real added value or merely consumes capacity and attention that could be more productively used elsewhere.
  4. Question: Is the community’s invested attention and engagement for this proposal appropriate, given the expected benefits and clarity of the outcomes?
    Evaluation Focus: Examines whether community energy is being used wisely—whether the referendum’s topic is significant and clear enough to justify broad participation and discussion.

■Transparency and Traceability

Evaluation Target: Addressing the question of whether the proposal enables evidence-based tracking and evaluation.

  1. Question: Does the proposal clearly define what specific change the community is signaling and why this concern is relevant?
    Evaluation Focus: Checks whether the signal’s goal and its significance for the network are clearly and understandably presented, so voters know exactly what they’re voting on.
  2. Question: Are the background, data, or analyses disclosed to justify the concern, enabling voters to make informed decisions?
    Evaluation Focus: Examines whether the proposer has presented facts, studies, or rational arguments to support the proposal, allowing voters to make an informed choice.
  3. Question: Is it transparently communicated how the referendum’s outcome will be used—e.g., as a basis for a follow-up referendum or a recommendation for developers or bodies?
    Evaluation Focus: Considers whether it’s clear what happens with the vote (since it’s non-binding): Does the community understand how their expression of will might be further processed or implemented?
  4. Question: Are clear criteria (e.g., minimum participation or required majority) specified to assess the outcome’s validity later?
    Evaluation Focus: Checks whether it’s defined when the signal is considered ‘strong’ or valid—e.g., whether a certain participation rate or clear sentiment is needed to serve as a basis for further steps.

■Track Record and Credibility

Evaluation Target: Addressing the question of whether the proposer(s) are credible and capable of meaningfully implementing the proposal.

  1. Question: Are the signal’s proposers known in the community for expertise in the relevant area, making their concern credible?
    Evaluation Focus: Examines whether the individuals behind the referendum have expertise or experience with the topic, making their motivation and assessment more credible.
  2. Question: Can the proposers point to previous successful governance initiatives or constructive contributions to the Polkadot ecosystem that bolster their credibility?
    Evaluation Focus: Looks at the initiators’ track record—whether they’ve acted responsibly and successfully in Polkadot governance or the community in the past.
  3. Question: What is the community’s sentiment regarding the proposers’ trustworthiness—do supportive voices or expressed concerns predominate?
    Evaluation Focus: Asks about the public perception of the initiators. If many community members express positive experiences or trust, it enhances credibility; widespread concerns suggest otherwise.
  4. Question: Does the proposers’ background indicate they could initiate follow-up actions or competently pursue the concern if the signal succeeds?
    Evaluation Focus: Checks whether the initiators have the connections, expertise, or influence to take concrete steps after the signal, advancing the issue effectively, which increases the signal’s impact.

GROUP 3: Spending Origins

Origins: [11]Treasurer, [30]SmallTipper, [31]BigTipper, [32]SmallSpender, [33]MediumSpender, [34]BigSpender

■Impact on the Ecosystem

Evaluation Target: Addressing the question of whether the proposal strategically and sustainably strengthens the network.

  1. Question: Does the proposal measurably contribute to the long-term development, adoption, resilience, or relevance of Polkadot?
    Evaluation Focus: Checks whether the proposal promotes key network success factors—e.g., technological advancement, user growth, resilience, or visibility.
  2. Question: What sustainable added value does the proposal bring to the Polkadot ecosystem in the long term, beyond the immediate project duration?
    Evaluation Focus: Assesses whether the proposal generates tangible, lasting benefits for the network beyond the project period.
  3. Question: Is an existing structural weakness addressed?
    Evaluation Focus: Asks whether the proposal specifically targets systemic weaknesses, delivering a sustainable development boost.
  4. Question: Does the proposal promote interoperability, user retention, or parachain development?
    Evaluation Focus: Evaluates strategic impact in key areas central to Polkadot’s scalability and attractiveness.

■Governance Compliance

Evaluation Target: Addressing the question of whether the proposal is appropriately contextualized.

  1. Question: Does the proposal clearly fall within the scope of the chosen origin (Treasury, Tipper, Spender)?
    Evaluation Focus: Checks whether the proposal’s content and goal are correctly assigned to the respective governance track.
  2. Question: Are there previous proposals with comparable content, and if so, what were their outcomes?
    Evaluation Focus: Examines whether experiences from similar predecessors support or question the process choice and the proposal’s likelihood of success.
  3. Question: Is the governance system being used meaningfully or burdened?
    Evaluation Focus: Assesses whether the concern is efficiently and appropriately placed in the OpenGov system—or whether it unnecessarily strains processes.

■Cost-Benefit Ratio

Evaluation Target: Addressing the question of how efficiently resources are used relative to the impact.

  1. Question: Is the requested amount proportionate to the potential or demonstrated benefit?
    Evaluation Focus: Assesses whether the financial effort is justified by a correspondingly high impact.
  2. Question: Is the budget framework reasonable compared to similar proposals?
    Evaluation Focus: Checks whether the requested budget aligns with market standards or is noticeably over/underestimated.
  3. Question: What specific added value does the Treasury or network gain in return for this expenditure?
    Evaluation Focus: Evaluates the expected return on Treasury—i.e., what benefit the community gains from the investment (e.g., users, tools, knowledge, efficiency gains).
  4. Question: Were cheaper alternatives considered?
    Evaluation Focus: Questions whether the proposers seriously considered possible alternatives before submitting the current proposal.

■Transparency and Traceability

Evaluation Target: Addressing the question of whether the proposal enables evidence-based tracking and evaluation.

  1. Question: Is it clearly communicated how and for what purposes funds will be used—including KPIs, milestones, metrics?
    Evaluation Focus: Assesses whether the proposal discloses precise and measurable plans for fund use and goal achievement.
  2. Question: Are budgets, timelines, and work packages clearly specified?
    Evaluation Focus: Examines whether the project plan is transparently and structurally presented, ensuring progress and fund use remain verifiable.
  3. Question: Are there success criteria for later evaluation?
    Evaluation Focus: Assesses whether measurable indicators have been defined to retrospectively evaluate project success.
  4. Question: Is documentation or reporting planned?
    Evaluation Focus: Checks whether transparent reports and project documentation are planned to enable continuous tracking.

■Track Record and Credibility

Evaluation Target: Addressing the question of whether the proposers are credible and capable of meaningfully implementing the proposal.

  1. Question: Have the proposers or involved organizations made verifiable, traceable contributions to the ecosystem?
    Evaluation Focus: Evaluates the proposers’ track record in the ecosystem (e.g., previous projects, open-source contributions, community activities).
  2. Question: What projects have been successfully implemented so far?
    Evaluation Focus: Examines specific past experiences that allow conclusions about implementation ability and reliability.
  3. Question: Are there publicly accessible references (e.g., code repositories, publications) or community feedback supporting the proposers’ credibility?
    Evaluation Focus: Assesses external evidence showing whether the proposers have trust, recognition, or professional qualifications.
  4. Question: Is the team capable of delivering the promised outcomes?
    Evaluation Focus: Checks whether the team is realistically able to successfully implement the proposal with appropriate quality.

GROUP 4: Referendum Management Origins

Origins: [20]ReferendumCanceller, [21]ReferendumKiller

■Impact on the Ecosystem

Evaluation Target: Addressing the question of whether the proposal strategically and sustainably strengthens the network.

  1. Question: Does canceling this referendum prevent potential harm or a misstep in the network, thereby preserving the ecosystem’s long-term stability and security?
    Evaluation Focus: Examines whether the intervention protects the network from negative consequences (e.g., technical issues or dangerous decisions), positively impacting network health.
  2. Question: Does this measure help maintain the network’s strategic direction by stopping an irrelevant or counterproductive initiative early?
    Evaluation Focus: Checks whether cancellation prevents the network from engaging with an issue that doesn’t advance it or distracts from essentials, preserving resources for strategically meaningful efforts.
  3. Question: Would continuing the referendum jeopardize the network’s sustainable development, and does cancellation counteract this potentially adverse outcome?
    Evaluation Focus: Considers whether the referendum—if continued or implemented—could harm the ecosystem long-term, and whether cancellation provides critical protection to ensure positive development.
  4. Question: Does the decision to cancel the referendum ultimately support the Polkadot ecosystem’s resilience and credibility by protecting the network from a harmful decision?
    Evaluation Focus: Questions whether cancellation strengthens trust in governance by demonstrating the system’s ability to protect itself from major errors—a key factor for a long-term robust ecosystem.

■Governance Compliance

Evaluation Target: Addressing the question of whether the proposal is appropriately contextualized.

  1. Question: Is canceling the referendum expressly permitted under current governance rules, and is this step factually justified by the specific circumstances?
    Evaluation Focus: Checks whether there is a regulatory basis for termination (e.g., in emergencies) and whether the current situation truly warrants such an exceptional intervention.
  2. Question: Is the responsible origin (Referendum Canceller/Killer) being used as intended—e.g., to stop a flawed or obsolete referendum—and not misused for other purposes?
    Evaluation Focus: Examines whether the special authority is applied correctly, only to address clear issues it was designed for, rather than potentially suppressing legitimate but unpopular initiatives.
  3. Question: Is the referendum’s termination consistent with past precedents or defined guidelines, or are there signs this measure is being used for the first time or abusively?
    Evaluation Focus: Considers whether similar cases were handled similarly in the past, supporting legitimacy, or whether this cancellation deviates, raising questions about governance compliance.

■Cost-Benefit Ratio

Evaluation Target: Addressing the question of how efficiently resources are used relative to the impact.

  1. Question: Does the benefit of protecting the network from a potentially harmful decision outweigh the costs in terms of possible loss of trust or questioning of the governance process?
    Evaluation Focus: Weighs whether protecting the network through cancellation is more important than the potential damage from participants losing trust or criticizing the process.
  2. Question: Does canceling render the community’s already invested voting effort obsolete, and is this justifiable compared to the harm prevented?
    Evaluation Focus: Examines the measure’s efficiency regarding prior investments: Many may have already voted or discussed (effort) that becomes void. Is this loss of effort justified by preventing greater harm?
  3. Question: Were less invasive alternatives available to address the flawed or unwanted referendum’s issue, and were these evaluated for their cost-benefit ratio?
    Evaluation Focus: Checks whether, instead of cancellation, options like correction, a counter-proposal, or letting the vote run its course existed, potentially achievable with less intervention and similar benefits.
  4. Question: Is the cancellation decision overall efficient, protecting the network from significantly greater risks or costs the referendum might have caused?
    Evaluation Focus: Assesses the overall economic sense of the intervention—whether preventing a potentially catastrophic outcome delivers far greater benefit and justifies the relatively smaller costs.

■Transparency and Traceability

Evaluation Target: Addressing the question of whether the proposal enables evidence-based tracking and evaluation.

  1. Question: Are the reasons for canceling the referendum clearly and comprehensibly explained, supported by concrete facts, analyses, or evidence of identified issues?
    Evaluation Focus: Checks whether the motivation for this drastic measure is openly communicated—e.g., by explaining what’s wrong or dangerous about the referendum.
  2. Question: Is it openly communicated who initiated the cancellation (individual or body) and on what basis the decision was made (e.g., technical findings, security concerns, or a governance resolution)?
    Evaluation Focus: Examines the chain of responsibility and information: The community should see who started the process and why.
  3. Question: Is it clear how the original concern of the canceled referendum will be handled moving forward (e.g., revision and resubmission or alternative solutions), so the community can follow the next steps?
    Evaluation Focus: Considers whether it’s communicated whether and how the referendum’s underlying issue will be addressed post-cancellation.
  4. Question: Are there documented records or minutes capturing the decision-making process and justification for cancellation, enabling later analysis and governance learning?
    Evaluation Focus: Checks whether the entire process—from issue identification to discussion and decision—is recorded in writing.

■Track Record and Credibility

Evaluation Target: Addressing the question of whether the initiators are credible and capable of meaningfully implementing the cancellation.

  1. Question: Can the initiators of the referendum cancellation demonstrate a verifiably responsible role in the Polkadot ecosystem, supporting their credibility in such decisions?
    Evaluation Focus: Examines whether those proposing cancellation have distinguished themselves in roles or actions showing responsibility and commitment to the network.
  2. Question: Have these actors previously undertaken comparable interventions, and did they prove justified and competent in doing so?
    Evaluation Focus: Looks at history: If the same individuals or bodies have canceled or intervened similarly before, the past outcome can indicate whether they acted prudently and successfully.
  3. Question: Do the proposers enjoy significant trust within the community, so their recommendation to cancel is seen as well-considered and in the network’s best interest?
    Evaluation Focus: Asks about general reputation, considering if community feedback supports their credibility.
  4. Question: Do the involved individuals or bodies have the necessary expertise and experience to accurately assess the situation and substantiate the need for this extraordinary measure?
    Evaluation Focus: Checks whether the initiators are competent to judge the referendum as problematic, taking into account both technical and governance-related expertise.

Voting Table.

Based on the analysis results, vonFlandern casts its vote in accordance with the Voting Table below:

VOTING OUTCOME VOTING
AYE ✅ ABSTAIN 🤷 NAY ❌ OVERALL VOTE CONVICTION AMOUNT (DOT | KSM)
AYE ABSTAIN NAY
5 0 0 AYE ✅ 5x 100% 0% 0%
4 1 0 AYE ✅ 4x 80% 0% 0%
4 0 1 AYE ✅ 3x 60% 0% 0%
3 1 1 AYE ✅ 2x 40% 0% 0%
3 2 0 AYE ✅ 3x 60% 0% 0%
3 0 2 AYE ✅ 1x 20% 0% 0%
2 3 0 ABSTAIN 🤷 0.1x 40% 60% 0%
2 2 1 ABSTAIN 🤷 0.1x 40% 40% 20%
2 1 2 ABSTAIN 🤷 0.1x 40% 20% 40%
1 4 0 ABSTAIN 🤷 0.1x 20% 80% 0%
1 3 1 ABSTAIN 🤷 0.1x 20% 60% 20%
1 2 2 ABSTAIN 🤷 0.1x 20% 40% 40%
0 5 0 ABSTAIN 🤷 0.1x 0% 100% 0%
0 4 1 ABSTAIN 🤷 0.1x 0% 80% 20%
0 3 2 ABSTAIN 🤷 0.1x 0% 60% 40%
2 0 3 NAY ❌ 1x 0% 0% 20%
1 1 3 NAY ❌ 2x 0% 0% 40%
1 0 4 NAY ❌ 3x 0% 0% 60%
0 2 3 NAY ❌ 3x 0% 0% 60%
0 1 4 NAY ❌ 4x 0% 0% 80%
0 0 5 NAY ❌ 5x 0% 0% 100%

Current maximum amounts for voting: 5,000 DOT | 200 KSM

Check our votes at subsquare: vonFlandern/VFDA | vonFlandern/VFKA