[Community-Discuss] AFRINIC Transfer Log format

[Owen DeLong]

Interesting…

In the $version dictionary, there are (among others) the following two attributes defined:
	“UTC_offset” — UTC Offset of the producer
	“production_date” — a datetime value when the file was produced.

What is not clear (at least to me) is whether the datetime values (there are two
other attributes  describing the start and end time covered in the file) are expressed
in localtime per the given UTC_offset or in some form of epoch reference (e.g.
a posix timestamp is seconds since the epoch which is currently widely defined
as midnight UTC 01 January 1970).

It’s also not clear in the case of local time in the field whether that is intended to
be current local time at production (in which case, one would expect a DST flag
attribute as well) or whether it is expected to always be local standard time
regardless of the application (or not) of the DST silliness in the locale in question.

Otherwise, it seems quite concise, readable, and useful as a definition allowing virtually
anyone to easily parse the resulting JSON.

Owen


> On Jul 22, 2019, at 05:21 , Daniel Shaw <daniel at techdad.xyz> wrote:

> 

> Hello all,

> 

> @Sunday,

> 

> Just to be clear: the OP, Raymond was asking if it could be CSV. It is

> already in JSON format. Not the other way around.

> 

> @Raymond,

> 

> The common JSON format used by all 5 RIRs is documented here:

> https://github.com/nro-ecg/transfer_log

> 

> In particular the comment schema is this:

> https://github.com/nro-ecg/transfer_log/blob/master/transfer_log.jcr

> 

> (the jcr language for describing json is explained here:

> http://codalogic.github.io/jcr/)

> 

> And an example single entry to look at is this:

> https://github.com/nro-ecg/transfer_log/blob/master/transfer_example.json

> 

> The full exlanation, including all the above links is found on the ARIN site:

> https://www.apnic.net/about-apnic/corporate-documents/documents/policy-development/transfer-log-format/

> 

> You'll find that all RIRs use this format, and in addition APNIC also

> publish the CSV format you like. My suggestion is that if you want to

> request AFRINIC staff to evaluate the viability of doinf the same,

> don't just throw it out on the community list. Rather reach out to

> staff directly. You should find details on https://afrinic.net/, under

> "contact us". Definitely raise it on the list as you have done to get

> input and comments from the community, as you also have done. But

> directing requests for staff at staff is usually more effective that

> relying on it being picked up on a large public list.

> 

> Going against my own advice above, staff, if you *are* reading this

> I'd suggest that before worry about any other formats you a) include

> the .jcr definition and/or a readme file in the

> http://ftp.afrinic.net/pub/stats/afrinic/transfers/ folder, so that

> this is self-documented like the extended delegated stats file is. and

> b) add a page in the web site similar to ARIN's about this file

> format.

> 

> Cheers!

> - Daniel

> 

> 

> 

> On Sun, 21 Jul 2019 at 14:27, Sunday Folayan <sfolayan at gmail.com> wrote:

>> 

>> I am sure the staff will be glad to give you their data dictionary.

>> 

>> Publishing in JSON format should not lose anything that will have been published in CSV format.

>> 

>> If the comments are not rich enough for you, You can start a collaborative project to fix this, and put it back into the community.

>> 

>> Nothing also stops you from pushing for a common RIR JSON specification, with your findings and use case. There may be others out there who may also need it.

>> 

>> Sunday

>> 

>> On Sun, Jul 21, 2019, 02:51 raymond <subs at retornam.com> wrote:

>>> 

>>> The problem isn't that JSON is hard to parse, my problem is that

>>> fields in the JSON returned are not

>>> documented and that different registries return different JSON

>>> formatted fields.

>>> 

>>> Also because JSON also doesn't allow  inline comments, I personally

>>> prefer a well commented CSV file over

>>> an un-documented JSON one.

>>> 

>>> /re

>>> 

>>> On Sat, Jul 20, 2019 at 5:52 PM Owen DeLong <owen at delong.com> wrote:

>>>> 

>>>> This is an operational matter for staff… Why would it need to go through the policy process?

>>>> 

>>>> The policy in the CPM does not and should not specify the reporting format.

>>>> 

>>>> I do find it interesting, however, that he considers JSON difficult to parse vs. CSV.

>>>> 

>>>> I find the opposite. There are many libraries available for parsing both formats, but the level of

>>>> consistency in the definition and syntax specification for JSON, especially around quoting and

>>>> handling of strings containing quote and delimiter characters plus the hierarchical dictionary

>>>> nature of JSON are, IMHO, a vastly superior format that is easier to parse while providing much

>>>> greater flexibility in the type of content delivered.

>>>> 

>>>> Owen

>>>> 

>>>> 

>>>> On Jul 20, 2019, at 16:54 , DANIEL NANGHAKA <dndannang at gmail.com> wrote:

>>>> 

>>>> Hi Raymond,

>>>> This is interesting, it has to go through a policy proposal. Policy proposal are discussed at AFRINIC meetings.

>>>> 

>>>> After discussion then it is passed for implementation.

>>>> 

>>>> Daniel K. Nanghaka

>>>> 

>>>> On Saturday, July 20, 2019, raymond <subs at retornam.com> wrote:

>>>>> 

>>>>> Hi,

>>>>> Would it be possible for AFRINIC to adopt the transfer log format used by APNIC?

>>>>> If so, who do I need to contact to make this happen? If not, can the fields in the JSON

>>>>> format be documented for reference?

>>>>> 

>>>>> APNIC has a well documented policy[1] on the  CSV logs they produce daily at [2]

>>>>> 

>>>>> [1] https://www.apnic.net/about-apnic/corporate-documents/documents/policy-development/transfer-log-format/

>>>>> [2] https://ftp.apnic.net/transfers/apnic/

>>>>> 

>>>>> APNICs format  stands out as it is easy to parse the data

>>>>> compared to the various JSON formats used by other registries(including AFRINICs)

>>>>> 

>>>>> RIPENCC https://ftp.ripe.net/pub/stats/ripencc/transfers/  JSON

>>>>> ARIN https://ftp.arin.net/pub/stats/arin/transfers/ JSON

>>>>> LACNIC https://ftp.lacnic.net/pub/stats/lacnic/transfers/ JSON

>>>>> AFRINIC http://ftp.afrinic.net/stats/afrinic/transfers/ JSON

>>>>> 

>>>>> 

>>>>> /re

>>>>> 

>>>> _______________________________________________

>>>> Community-Discuss mailing list

>>>> Community-Discuss at afrinic.net

>>>> https://lists.afrinic.net/mailman/listinfo/community-discuss

>>>> 

>>>> 

>>> 

>>> _______________________________________________

>>> Community-Discuss mailing list

>>> Community-Discuss at afrinic.net

>>> https://lists.afrinic.net/mailman/listinfo/community-discuss

>> 

>> _______________________________________________

>> Community-Discuss mailing list

>> Community-Discuss at afrinic.net

>> https://lists.afrinic.net/mailman/listinfo/community-discuss

> 

> _______________________________________________

> Community-Discuss mailing list

> Community-Discuss at afrinic.net

> https://lists.afrinic.net/mailman/listinfo/community-discuss