[rpd] impact analysis for policies

Mon Jun 24 14:03:20 UTC 2019

Hi Sylvain,

You’re good man! You just convinced me and I’m not easy ;-)

I accept your bet ;-) Let’s have *now* a very simple policy proposal for making the impact analysis timely mandatory.

I’m proposing you the following and I’m doing it openly in the list, so others can provide inputs as well.

According “3.6 Varying the Process”, for that we need a *co-chair* to accept it. Any taker?

This is the variance that I’m proposing:
Publish officially the policy proposal and start a 4-weeks for community discussion.
If after that, the chairs believe there is consensus, last-call for 4-weeks.
If there is no-consensus, allow to repeat the process, maximum up to one month before the next meeting (then there is no need for the variance of the process).
***

ACTUAL text (at 3.4.1):

The Working Group Chair(s) may request AFRINIC to provide an analysis (technical, financial, legal or other), of the impact of the draft policy proposal.

DRAFT proposed text (replace the previous text):

AFRINIC will publish an impact analysis (technical, financial, legal or other) at least 10 days before the next Public Policy Meeting. If justified, in case of extremely complex new policy proposals, AFRINIC will publish at least a draft version of the impact analysis.

***

This small change in our PDP will prove ourselves that AFRINIC community is able to pro-actively discuss and pass policy proposals in the list.

What do you think?

Regards,

Jordi

@jordipalet

El 24/6/19 15:20, "Sylvain BAYA" <abscoco at gmail.com> escribió:

{Warning & apologize :-) you might found it tl;tr}

Hi all,

Le 6/23/2019 à 11:55 AM, JORDI PALET MARTINEZ via RPD a écrit :

Hi Sylvain,

Again thanks for your inputs. Comments below in-line.

Regards,

Jordi

@jordipalet

El 22/6/19 11:16, "Sylvain BAYA" <abscoco at gmail.com> escribió:

[...] 

CPM section 3.4.1 : «[...]The Working Group Chair(s) may request AFRINIC to provide an analysis (technical, financial, legal or other), of the impact of the draft policy proposal.[...]»

...i think we must follow our rules or change them if they appear to be broken somewhere. Right ?

There are two things here. One is the policy manual and the other one is the internal AFRINIC operational procedures.

Yes of course, i mean i *expect* the "internal AFRINIC operational procedures" to folow the CPM (Consolidated Policy Manual). Am i wrong ?

If you read the message from Dewole on the thread you referred to (https://lists.afrinic.net/pipermail/rpd/2019/009033.html) it clearly stated that this is the procedure:

Ok i have read it again carefully, thanks for have pointing it to me.

What i understand, from the explanation of Dewole, is that :

* The Chairs and the Staff found an arrangement regarding the production and publication of impact analysis reports ;

* This *agreement* was concluded by them to simplify the process (PDP) and the procedure (Operations) ; 

* The Staff has accepted to provide (in due time) an impact analysis for every versions of all the policy proposals in discussions ;

* The Chairs will no longer have to request again for a specific impact analysis report when needed ;

* We have a kind of automated sub-system.

...my conclusions in correlation with the facts you provided below :

+ The Chairs have got a willing|desirable arrangment with Staff about impact analysis publication but ;

+ The Staff has proved to be unable to provide impact analysis for all the versions of the policy proposals in due time (they can certainly explain this fact) ;

+ The Staff are *always* providing impact analysis report for a random set of policy proposals and not for all versions of a given policy proposal ;

+ Sometime they have failed to provide very needed impact analysis reports at time ;

+ An analysis (yours) for all the policy proposals shows that : the actual procedure the Staff is using to provide Impact Analysis report looks as producing random results (and this is not better than the normal Process [PDP] as provisioned by CPM section 3.4.1) ;

+ The *agreement* obtained by the Chairs is proven to be flawed, particularly by your analysis generously (thanks) provided below ;

+ By this *agreement*, the Chairs have lost|abandoned (involuntarily for sure) their responsibility in regard of the impact analysis report requests ; 

+ It seems to be an opaque procedure because community was not aware before the recent mail from Dewole (*Varyance* [CPM section 3.4.1] is a good|useful tool, but we deserve a bit of transparency and to be timely informed of such a *emergency* [and *temporary*] decision).

CPM section 3.6 : «3.6   Varying the Process 

The process outlined in this document may vary in the case of an emergency. Variance is for use when a one-time waiving of some provision of this document is required.

1.       The decision to vary the process is taken by a Working Group Chair.

2.       There must be an explanation about why the variance is needed.

3.       The review period, including the Last Call, shall not be less than four weeks.

4.       If there is consensus, the policy is approved and it must be presented at the next Public Policy Meeting.»

I prefer that we follow (or openly amend) the CPM section 3.4.1, and hopefully we will have more chance to receive the requested impact analysis reports in due time.

Perhaps we should also pass through a new policy proposal to ameliorate something in regard of that...not sure what's useful to do but :-/

“For a while now, the procedure (as part of lessons learnt) has been for 
staff to provide their analysis of draft proposals once they are 

received so it's not really on the author. Co-chairs and staff have long 

agreed that this makes sense.”

...IMHO it was a very good deal for the community, but it is now proven to badly functioning since then. 

I guess we want to fix. Then what to do ?

Your comments on the disponibility of impact analysis reports for all the actual policy proposals seems to confirm a broken procedure ; where the PDP is not applyed (with this agreement the Chairs are not doing their duty : ask for a given *needed* impact analysis report).

I fully agree with the rest of the text, that mentions that if a new version is sent, for example one week before the meeting, it may not be feasible. However, my point is that, proposals that have been received several weeks before the meeting AND had already a previous impact analysis, AND the authors just updated the proposal to comply with the impact analysis, in my opinion, results in a few hours (not to say minutes) to rework the previous impact analysis.

...useful insights for the Staff, i think ? 

Chairs and staff know, that at least in my case, if I get an email from them with questions about my proposal version or impact analysis, I do my best to respond in a matter of few hours, often during the next hour or so, unless it falls during the time I’m traveling or sleeping, but most of the time I respond within the same day. I don’t break during the weekends, don’t do holidays or take days off, and average sleep 5 hours per day, so it is difficult to believe that I’m not responding “quickly”, unless by chance an email is classified as spam and filtered (I don’t recall it happened with emails from staff and co-chairs, but just in case).

Also take the considerations from SM about the existence of similar policies and impact analysis in other RIRs, which may be at least a starting point ( https://lists.afrinic.net/pipermail/rpd/2019/009036.html).

...thanks Jordi.

You must know that you are helping me a lot to quickly accomodate to the PDP and particulary some hidden (but don't get me wrong it's just about facts we are still discussing), and *policy-free*, practices in this PDWG ? 

I learnt and concluded two things while reading, again, the mentioned SM's mail carefully :

+ To ameliorate the, timely, delivering of impact analysis reports, we need (amongst other actions) to figure out : "How long does it usually take from the time the PDWG is asked to provide feedback [...] to when the analysis of the proposal is available ? 

You did a bit of analysis, of the situation, below and it was really instructive to me. Thanks once more. Hope we use it to fix the issue.

+ While "all the analysis work is being left to Afrinic Ltd." Are we expecting to have an optimal PDP ? 

So, if we analyze *all* the policy proposals on the table (so no just “defending” my own position, but also other authors of proposals):

Thanks for sharing your thoughts (analysis) about the disponibility of impact analysis for all the policy proposals on the table. 

Hopefuly it will help the staff to quickly provide the missing ones...

1)      AFPUB-2019-GEN-001-DRAFT01: published on 17th May. This is the only proposal, which is quite complex, that I agree may take longer than 4 weeks for the impact analysis, even if there is already an existing impact analysis in other RIRs.

2)      AFPUB-2019-IPv4-001-DRAFT01 and AFPUB-2019-IPv4-002-DRAFT01: published on 13th May. However, those proposals *share* the same impact analysis as the one already done for AFPUB-2018-GEN-003-DRAFT01 (1st November 2018) and they are *simpler* than that one, take only on IPv4 (instead of IPv4+IPv6+ASN), impact analysis already available also from other RIRs.

3)      AFPUB-2019-V6-001-DRAFT01: published on 14th May. Impact analysis already published for the previous version. It is true that this policy proposal is a bit more complex than the previous versions, but I think it was clear in the meeting (where it reached consensus), that the staff don’t have any “impacting” considerations, so what they said in the meeting, could had been published before.

4)      AFPUB-2019-ASN-DRAFT02: published on 11th April. My god! How come we can’t have in more than 2 months an impact analysis, which was already published for the previous version, while this version is actually precisely, much simpler, considered the impact analysis to correct all the points, and *reached consensus* during the discussion in the list? I’m tempted to appeal the chairs decisions on it (no-consensus), because if we had the impact analysis and the chairs, really considered the list, there was overwhelming support and, in my opinion, none of the comments in the meeting was actually sufficiently substantiated as something *against* the arguments in favor. I’m not going to do so, because I support the previous co-chairs decisions, and unfortunately (and this is BIG BIG BIG issue), some of the inputs of the people in the room where from folks that either a) they are not in the mailing list (I already suggested them to join and participate), b) some folks, which are in the list, which are *actual* authors of other policy proposals, didn’t brought their inputs in the list, they have actually brought them in the meeting, just for the *insane sake of overthrow this proposal* and exclusively for their own interest (as I’m not supporting their own policy proposals. I those co-authors of other policy proposals are really looking for the well of the community, they *MUST* participate in the list *not only* to defend their own policy proposals, but also to improve the ones from other authors (as I do). If they provided those inputs in the list, then I could have made a new version in time for resolving their issues and reaching consensus.

5)      AFPUB-2018-GEN-004-DRAFT02: published on 18th April. Received substantial inputs in the list, but it was withdrawn for author personal reasons (I know the details, but I will suggest him to explain to the list ASAP). However, despite having a previous impact analysis, we got nothing this time. How come?

6)      AFPUB-2016-GEN-001-DRAFT08: published on 7th June. There was a previous version impact analysis. The proposal didn’t change so much that that impact analysis could have been reviewed, at least in draft, to have “something” before the meeting. So again, why not we had something?

7)      AFPUB-2018-GEN-001-DRAFT03: published on 5th June. Again, there was a previous version impact analysis, and the changes were to accommodate to them, making the proposal much simpler, and ensuring that those details on the impact analysis are resolved. So how come we didn’t have one at least in draft? This one also has been adopted in other regions, and of course, we have impact analysis in others.

8)      AFPUB-2018-v4-001-DRAFT-01: published on 28th October (2018), is the only one that had an impact analysis. Is this meaning that staff needs 8 months to complete an impact analysis? *by the way* I don’t know when it was completed, because a) I can’t find any message announcing it to the list, b) the impact analysis doesn’t say the date! Is that right?

9)      AFPUB-2017-GEN-002-DRAFT-05: published on 9th June. There is a previous version impact analysis, and again, there are minor changes, so I could have expected at least a draft impact analysis considering those minor changes.

...randomized delivering ?

Is it intended that it walks like that  ?

Note that sometimes the publication is delayed several days, so the staff has got in fact *more* days to start the impact analysis.

Note also, that in some cases, if an impact analysis can’t be completed in time, it is good enough to have the same “draft” inputs that the staff is providing in the meeting, *but* authors need that *before* the meeting, so we can work out a clearer explanation in case of discrepancies.

:-) Jordy, reassures me that we can correct it...and please tell me also how we can succeed ?

Any other voice out there ? :'-(

I may agree that if a proposal has been submitted only 10 days before the meeting, it may be difficult to make it in time, but not for a “new” version of an existing proposal, which already had the analysis impact.

Ok if it's unpractical, we should try to modify the CPM, at section 3.4.2, to fix|ameliorate what seems to be broken... 

CPM section 3.4.2 : «[...]No change can be made to a draft policy within one week of the meeting. This is so that a stable version of the draft policy can be considered at the meeting.[...]» 

As my analysis above shows, I don’t think anything is broken. We can always improve the PDP, but if something is already part of the operational process, it is just a clarification, and there are more important policy proposals in the table than sending on for just resolving a non-problem. I will be in favor of this if we have 2 days policy-day next time!

*** Chairs and staff must agree on how much an impact analysis may take in every policy proposal case. Some proposals may take hours. Very few may take weeks or even more than a month. However, it may be a *management* or even *board* problem to ensure that the staff has available “human resources” to do their work timely. May be the folks in the staff that are responsible for making the impact analysis (I know it implies different departments and even CEO approval), need to prioritize, up-front of a meeting, what of their tasks are more important for a successful policy-day.

I agree that the BoD could help here; but i think we must first clearly state the problem to solve.

First, our Chairs *agreement* with the Staff, creates an unexpected situation. (this is fact)

Then, to handle this fact, IMHO we should stop this *agreement* and if we want to ameliorate things in that regard, it will be possible to try something new (BoD reaction, PDP revision, other possibility)...

And of course, I’ve asked the staff several times for it, during the last weeks. Co-chairs were copied. 

Ok, i personaly think that it is legitimate to want to move things forward ; but i also understand (policily speaking) that the staff was in their rights to not considere your *direct* request as a legitimate one. Even though i guess they were just busy :-) 

Agree (see previous comment), but one of the *most* important things that a RIR must do is PDP! This is top priority.

Ok, hope that the coming facts will convince us of that *top* priority...

Precisely, most of the newer versions, at least in the case of my proposals, where to address the points raised in the impact analysis. So, it should be easy to get the previous impact analysis, compare the differences of the old and new versions of the policy proposal, against the impact analysis, and raise if something is missing.

:-) i'm sure that at least one of them (policy liaison?) might be reading...

Do you think we should also (policily speaking) ask one of them (say the same staff member i mentioned above) to follow|read the discussions and react when we need|seek their expertise ? :-D

I know by the fact, they are reading, that’s why I post those messages here when private ones didn’t work as expected! Again, not blaming them, just makes sure we improve things.

...again, not blaming anyone too, i just want to understand how things walk ; in order to see if we can ameliorate something.

Then, tell me please : why (despite the provision of CPM section 3.4.0) we don't get any response since we ping them ? [1]

Staff and chairs know very well that I send them emails any time I can offer suggestions, ideas to improve participation, requests for anything which is good for them and for the community. I always try my best, because when you contribute, you discover issues to resolve. Nobody is perfect, this is about *collaboration from all*.

For example, I discovered a few weeks ago, that the numbering system of the policy proposal is flawed so suggested it to get changed and don’t differentiate them, just “year-number-version”. The point is that because the policy proposals get “classified” as “general, IPv4, IPv6 or ASN”, if during the discussion, the policy proposal change its scope, which happens sometimes, then you need to withdraw the policy proposal and re-start it. This was the reason instead of just updating my last year Intra-RIR policy proposal (which was “GEN”), turned into a new one.

Thank you for sharing your findings with us.

When i see the provision from the CPM section 3.4.1 then i confirm that the Staff has the full rights to change it right now :-)

CPM section 3.4.1 : «[...] Each draft policy is assigned a unique identifier by AFRINIC and the AFRINIC website shall also contain the version history and the status of all proposals. [...]»

Hopefully this is considered by the staff soon and we remove this classification soon, and we get all the policy proposal in the table renumbered to the “new” system.

Many thanks for your endeavor. I look forward to see that improvement.

Believe me, in most of the proposals, this is a matter of few hours of dedication. My only concern is if staff is overbooked, which is something that should be evaluated by the organization (not authors). 

First, have we already tried to fix the problem with our means (the policy) ?

As said before, there are two parts. PDP (policies) and procedures. Not all requires policies. If we micro-manage, it becomes cumbersome. In my thinking, we should use policies only when procedures don’t improve by themselves (or the community radically disagree with the procedures), or don’t exist at all. PDP is heavy, its slow, it requires a lot of community effort, which sometimes is not needed if just improving procedures is sufficient. Just consider the time it requires to have 10 policy proposals presented in the meeting. How if we try to present 15 or 20?

Thanks, i will remember this PDP picture you painted for us (me). :-)

[...] 

...it seems as there is something to fix :-)

Considering what i have said above, do you admit that, as it is, the process is flaw ?

Please, consider my point about the difference between operational procedures and policies. Micro-management is not good, even if sometimes is needed.

...i understand it thanks. 

However, my will is not to micro-manage operational tasks but to find appropriate way forward to deal with what seems to be flawed.

But if you know how to fix it without any PDP alteration. Fix it please.

I really appreciate your support, i see you trying to help me :-)

...just remember that i'm still learning the PDP and logically, i first rely on the CPM (v.1.3).

If some operational procedure doesn't follow the PDP, it shall be more difficult for me to understand how things walk :'-(

The impact for this is really big. Right now, we have 8 policy proposals that didn't reached consensus and only considering new proposal that I've in mind to submit (5), it means that for the next meeting we will have *at least* 13 policy proposals in the table.

...other might also propose drafts for new policy proposals :'-(

I hope so, more people contributing, and yes, that’s why I said “at least” :-)

:-) but please withdraw one of your two similar policy proposals about out of AFRINIC region resource (IPv4) transfers. That action could diminish the number of policy proposals ;-)

Remember that I presented them jointly, with the same time for two proposals, that for a single one, so actually that’s not taking time.  Anyway, if we can have a “clear” decision in the list from now to the next meeting, I’m happy to withdraw one of them, but this is only going to work if we have 200 voices in the list supporting one of those. Otherwise, I can withdraw the wrong one and then in the meeting, the participants decision withdraws the other one, so we have nothing. You see the point? 

...so you don't want to take the *risk* (i don't actualy see a risk, though) ? :-)

I may change my mind on this depending on how the discussion goes in the next weeks … Let’t talk that in the relevant thread!

Thanks again!

...no please, it's me :-)

__

[1]: https://lists.afrinic.net/pipermail/rpd/2019/009561.html

Shalom,

--sb.

[...] 
-- 

Regards,
Sylvain B.
<http://www.chretiennement.org> 
__
Website : <https://www.cmnog.cm>
Wiki : <https://www.cmnog.cm/dokuwiki>
Surveys : <https://survey.cmnog.cm>
Subscribe to Mailing List : <https://lists.cmnog.cm/mailman/listinfo/cmnog/>
Mailing List's Archives : <https://lists.cmnog.cm/pipermail/cmnog/>
Last Event's Feed : <https://twitter.com/hashtag/cmNOGlab3>
<https://twitter.com/cmN0G>
<https://facebook.com/cmNOG>
<https://twitter.com/hashtag/REBOOTcmNOG>
<https://twitter.com/hashtag/cmNOG>
<https://cmnog.wordpress.com/>
_______________________________________________ RPD mailing list RPD at afrinic.net https://lists.afrinic.net/mailman/listinfo/rpd 

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.afrinic.net/pipermail/rpd/attachments/20190624/44045539/attachment-0001.html>