[AfriNIC-rpd] abuse contact information in whois database (AFPUB-2010-GEN-002)

[Owen DeLong]

On Jun 16, 2010, at 11:21 AM, Tobias Knecht wrote:

> 
>>> Another request by members was that there should be the possibility to
>>> add some kind of object that contains 2 email addresses. One for
>>> personal interaction and one for automatic abuse reports. My idea was to
>>> ask for a abuse-mailbox attribute within the IRT Object. So that would
>>> fix that request.
>>> 
>> In ARIN, you can have zero or more abuse-c contacts per object.
>> I don't see why that couldn't work here.
>> 
>> Frankly, however, I don't think that will work as well as some people hope
>> because I have limited faith in the automated reporting systems following
>> the requested convention. Someone will think that the personal response
>> box is more responsive than the automatic reporting box and decide that
>> their need for response to automated reports outweighs the providers
>> need to sort things into different boxes. Others will simply report to both.
> 
> This is not a problem of this proposal. This is a problem of the over
> all situation at RIRs. We are having the problem of not having dedicated
> abuse mailboxes, we are having the problem of data accuracy, we are
> having the problem of complicated ways of parsing and finding abuse
> contacts.
> 
No, this is a problem of human nature, education, and experience.

Human nature is that rather than follow the proper prescribed process
and sending automatic abuse reports to automated-abuse-c and sending
personal reports to personal-abuse-c, what usually happens instead is
that both forms of abuse reports go to:

admin-c
tech-c
every-other-c
abuse-c
parent-admin-c
parent-tech-c
parent-abuse-c
parent-every-other-c
parents-parent-admin-c
etc.

The education problem comes in the form of the majority of the people
that exhibit the above behavior do so out of ignorance because they
have not been taught and do not bother to educate themselves on
the proper way to go about reporting abuse.

The experience problem comes from the fact that less than stellar
behavior from some providers in response to proper abuse reporting
has created anecdotal evidence and experience-based perception
that it is more effective to shotgun everyone with your abuse report.

> These are all self made problems.
> 
To some extent, perhaps. However, to a much larger extent, these
are problems collectively made by the systemic application of
human nature to imperfect systems built  by humans.

> This is not the last proposal you will see from us, because we want to
> change something.
> 
I have no doubt of that.

> The problem you described would not exist, if RIRs would have mandatory
> abuse contacts in their whois, offering these in an easy to use DNS
> based service, where everybody could ask for the right contact
> information and shutting restricting queries more and more, so only real
> messages will be sent to the personal accounts of the abuse desks.
> 
On this we will probably need to agree to disagree completely and vigorously.
I don't see any way in which mandatory abuse contacts in whois will change
the human nature problem, the educational problem, or even the experience-
based problem I described.

I do see several ways in which having an available field for an abuse contact
can in part help the situation. I don't see significant benefit to making it
mandatory, and, I do see a few potential downsides.

> That is exactly the way we want to go down. But we should start
> somewhere. And the problem that generates most pain at RIR members are
> the not existing abuse contact objects at the moment.
> 
I guess that depends on who you consider to be the RIR member. I think
what you are describing is the thing that causes the most pain at the
abuse reporter side. I would posit that most RIR members are actually
abuse report recipients rather than abuse report generators.

> The problem you described is absolutely real, but it is already today
> and not a problem of this proposal. So let talk about a dedicated abuse
> contact object and talk later about the other problem.
> 
You brought up the recommendation of separate abuse contacts for
automated vs. personal abuse reports. I responded to that subject.

> 
>>> How would you do that with already existing objects? Lets think about
>>> the abuse-mailbox attribute which can be part of every person or role
>>> object. What I do not want to happen that there will be more than one
>>> objects having an abuse-mailbox attribute. That would start the same
>>> trouble we have a RIPE at the moment, that there are 15 possibilities to
>>> put an abuse@ address and in some queries you find 10 of them with
>>> different addresses.
>>> 
>> Yep.  At ARIN, abuse-c is a field in an ORG object which can either be
>> inherited by all child objects, or, over-ridden by an abuse-c field in
>> a child NETWORK or ASN object. You can have zero or more abuse-c
>> fields in each such object. Each abuse-c field contains a handle for
>> a CONTACT object.
>> 
>> A CONTACT object is either a person or a role account.
>> 
>>> So would it be able to create a special role object that has a mandatory
>>> abuse-mailbox attribute, what I think makes sense, that can only be
>>> linked to an abuse-c?
>>> 
>> Why would it need to be a special role object? Why not just use the
>> generic CONTACT object that already exists and link to it the same
>> way that admin-c or tech-c links to it.
>> 
>> For example, say you have CONTACT objects as follows:
>> 
>> OD19-ARIN		Person
>> ABXX-ARIN		Role
>> 
>> Then, an ORG object, for example, could have the following
>> attributes:
>> 
>> tech-c: OD19-ARIN
>> admin-c: OD19-ARIN
>> abuse-c: ABXX-ARIN
>> 
>> Or, I could have ABXX-ARIN in all three fields, or, OD19-ARIN, or
>> any other combination. If I wanted, I could create yet another
>> CONTACT object for any of the fields.  Simple, easy to understand,
>> and pretty flexible, no?
> 
> This would work. Absolutely.
> If this is the solution AfriNIC members can live with, fine. I would
> suggest adding the abuse mailbox field and the RIPE equivalent "whois
> -b" option for short data output and unrestricted query access.
> 
I'm not sure what you mean by adding the abuse mailbox field.  Each
contact object has a required "Email" field. For example,
OD19-ARIN expands to:

[owen-delongs-macbook-pro:he.net/ipv6/business_site] owen% whois -h whois.arin.net OD19-ARIN

Name:       DeLong, Owen
Handle:     OD19-ARIN
Company:
Address:    c/o 3251 Firth Way
City:       San Jose
StateProv:  CA
PostalCode: 95121
Country:    US
Comment:
RegDate:
Updated:    2009-09-27
Phone:      +1-408-890-7992  (Office)
Email:      owen at delong.com

> Restrict abuse-c to one object. Instead we are running in the same
> problem than RIPE a few years ago.
> 
Again, I'm not sure what you mean by this. Do you mean a record should
have exactly one abuse-c or do you mean that abuse-c should only appear
in number resource records and not org records, or what?

> I would also like to see that the abuse-mailbox attribute is just shown
> if the object is linked to the abuse-c, because if not, you will have 3
> different admin-c, tech-c and abuse-c objects with 3 different abuse
> contacts. And that will end up in the same problem RIPE has today.
> 
I think you are not understanding.  Let me give a real example:

ORG Record:

OrgName:    DeLong Consulting
OrgID:      DELONG-Z
Address:    c/o 3251 Firth Way
City:       San Jose
StateProv:  CA
PostalCode: 95121
Country:    US
Comment:
RegDate:    2009-02-04
Updated:    2009-02-09
AdminHandle: OD19-ARIN
TechHandle: OD19-ARIN
AbuseHandle: OD19-ARIN

(yes, in this case, I'm using one CONTACT record for all three roles, but,
I suspect you understand that each role could have any CONTACT record
associated, and, you can have 1-n each of AdminHandle and TechHandle,
and 0-n of AbuseHandle).

Here's an example of a network resource with the above ORG as its
parent:

OrgID:      DELONG-Z
NetRange:   192.159.10.0 - 192.159.10.255
CIDR:       192.159.10.0/24
NetName:    DELONG
NetHandle:  NET-192-159-10-0-1
Parent:     NET-192-0-0-0-0
NetType:    Direct Assignment
NameServer: NS.DELONG.SJ.CA.US
NameServer: ASILOMAR.DELONG.COM
Comment:
RegDate:    1994-11-02
Updated:    2009-02-10

Now, in this record, you'll note no Admin/Tech/Abuse handles.
It could have them, but, since they are not present, it inherits
these properties from DELONG-Z.  Whois will display
them if you query for the network record, but, what is displayed
is stored in the parent record, not the resource record. You can,
optionally have contacts specified in the resource as well (0 or more
of each of Admin, Tech, Abuse) which, if present will be displayed.
The ORG data is also displayed by ARIN whois.

So, even though Abuse-c would point to a contact record in the
database, the whois interface would retrieve and display that
contact record along side the resource record. Does that meet
your need?

> 
>>> 15 million messages come from around 10 million uniq IP addresses.
>>> 
>> You're telling me that there are 10 million NEW spam source addresses
>> every day? I find that fairly hard to believe. Even so, I bet those 10 million
>> IP addresses don't represent 10 million unique NETWORK objects.
>> I don't think it takes lots of hours to store what you already retrieved
>> from whois in a hash with a time-stamp. The code for doing this is
>> well known and I suspect there are open source tools already available
>> that do so.
> 
> Nope sorry, that was not the way I wanted to say.
> 15 million trap hits per month coming from 10 million unique IP
> addresses per month. We see 10 million different IP addresses every
> month. Which is not the same as infected systems, because over an month
> they switch addresses.
> 
Understood.

> We are hashing this and we are offering our cache system to others to
> give them more easy access to these abuse contact information, because
> we know it is not that easy to get all the needed information out of
> whois that quickly.
> 
OK

> The problem is, that at the moment there is no easy way to query
> different whois servers and get the right answer, so everybody is doing
> it a little bit different and that is the problem.
> 
OK, but, this isn't the problem of the data structure. It is a problem of the
available UI to retrieve the data. You are proposing a change to the
structure of the data to address a problem with the UI. I agree that
there is a need for resolving your UI problem, but, I don't agree that
such a change to the underlying data structure is needed, nor do I see
doing so as a particularly reliable way to achieve the UI change that you
seek.

> There is the need to have a standardized way of searching and finding
> the right information.
> 
Agreed, however, that is a UI question...

> That is another point that is pointing to the IRT Object. ;-)
> 
And this is a data structure change, not necessarily a UI change.

> 
>>> But I think we can go down another road with issue. If we can not find
>>> consensus on the IRT Object and are not able to stop the query
>>> restrictions, I would propose to start a service like this:
>>> http://labs.ripe.net/content/updated-heuristics-abuse-finder-service
>>> 
>> That seems fairly reasonable and I suspect would be relatively trivial
>> to implement.
> 
> As mentioned above. Mandatory abuse contact : DNS based abuse contact DB
> : setting up a local DNS cache : all good.
> 
Yeah, not so convinced on that. I think that optional abuse contacts added
to whois with provisions for relaxed whois query rate limits where needed
is a viable alternative. I still don't see the case for making abuse contact
mandatory, and, I really don't think that DNS is the right place to store the
abuse contact information.

> That would make things easy and stops thousands of different ways to do it.
> 
Now, you are dreaming. It _MIGHT_ provide one consistent way you can
depend on, but, others will continue to use other methods.

>>> That would probably be more easy than it is at RIPE, because if there is
>>> only one object containing the abuse contact information it should be
>>> really easy to get this done.
>>> 
>> You might have to still look at the NETWORK or ASN object and the parent
>> ORG object, if the child object doesn't contain an abuse-c, but, you should
>> never need more than two queries to build your list of contacts for a given
>> address range. After that, it's just a matter of querying the applicable
>> CONTACT objects.
> 
> Absolutely right. But why penetrate the whois server if there would be
> another alternative?
> 
Because the other alternative requires additional maintenance steps on the
part of the end users that are likely to be poorly maintained and thus suffer
accuracy reduction over time.

>>>>> So in my opinion there is a need of unrestricted query access.
>>>>> 
>>>> We can, perhaps, agree to disagree on this. To me, it sounds like a problem
>>>> of application optimization which I would argue should not be used to place
>>>> additional undue burden on the RIR whois services.
>>> 
>>> Okay lets agree on that. ;-)
>>> 
>> Fair enough. Increasing the query rate limits or placing finer granularity
>> on them, btw, is not necessarily a bad idea in my opinion. Allowing you
>> to sign up an address for a higher query rate service with a signed
>> contract containing an AUP specifying you won't be harvesting the
>> data for unapproved purposes would also be a reasonable alternative.
> 
> DNS Based list for everybody?
> 
Certainly would not be my first choice.

Owen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.afrinic.net/pipermail/rpd/attachments/20100616/69ffbcca/attachment.html>