Search RPD Archives
[rpd] Cloud Innovation Displays Very Poor, If Not Criminal, Netizenship
Arnaud AMELINA
amelnaud at gmail.com
Mon May 25 01:28:10 UTC 2020
Hello, community
+1 @Gregoire and @Mark Tinka
*cloud innovation* were allocated *big bunch of IPv4* space as a *LIR*
with *no ASN*. Interesting, and no *v6*
While the bylaws defines LIR as followed:
++++++
Local Internet Registry (LIR):
any Network Operator that provides Internet services to distinct end-users
and end-sites
++++++
I wonder which network does cloud innovation operate and which internet
services it provides to end-users and end-sites in *Africa*.
How does this network *managing 3 x /11 of IPv4* *operate*?
There is something here for the community to learn about.
Regards
--
Arnaud
Le sam. 23 mai 2020 à 14:20, Gregoire EHOUMI via RPD <rpd at afrinic.net> a
écrit :
> Hello,
>
> Thanks Mark for exposing the details of the SEACOM AS37353 hijacking.
>
> I carefully read your report and also the Cloud Innovation Limited quick
> response including their attachments as justifications.
>
> I note that;
>
> ⁃ the service contract with Cloud Innovation covering the announcement of
> their prefixes by SEACOM AS37353 was terminated by SEACOM.
>
> ⁃ some stale IRR route objects existed after termination of the contract.
>
> ⁃ through some multiple layer distribution an organisation in Manila
> Philippines was “delegated“ an IP block from Cloud Innovation address space.
>
> ⁃ both upstream ISP and the customer in Manila set up a BGP session using
> SEACOM's AS37353 as the ASN of the Manila customer.
>
> ⁃ there was a prompt reaction from the involved parties that included
> apologies to SEACOM and the wider internet community.
>
> ⁃ there were promises from said parties to be a better netizen which would
> mean, them not hijacking other networks ASN's.
>
> ⁃ there was clear refusal to disclose the details of the customer in
> Manila Philippines who hijacked the affected SEACOM ASN.
>
> All put together, demonstrates that what happened was an impersonation and
> not a BGP configuration error, nor an oversight in checking the right to
> use of the SEACOM ASN.
>
> 1. Why is it that the real customer did not bother presenting its
> apologies to the community
>
> 2. Why is there refusal to reveal customer’s details?
>
> 3. Why is it that the said prefix is no longer seen in the routing table
> originated by the genius ASN or any other ASN?
>
> 4. Which networks were involved and what happened to the end users?
>
> Can someone from AFRINIC explain what “delegation of IP block” mean?
>
> Note: The self organised Internet knows how to deal with bad net citizens.!
>
> Best regards
> Gregoire Ehoumi
>
>
> -------- Original message --------
> From: Lu Heng <h.lu at anytimechinese.com>
> Date: 2020-05-09 5:43 a.m. (GMT-05:00)
> To: Mark Tinka <mark.tinka at seacom.mu>
> Cc: "rpd at afrinic.net >> AfriNIC Resource Policy Discussion List" <
> rpd at afrinic.net>
> Subject: Re: [rpd] Cloud Innovation Displays Very Poor, If Not Criminal,
> Netizenship
>
> To whom it may concern,
>
> On May 8, Mark Think posted a claim to multiple lists that Cloud
> Innovation was abusing an ASN (37353) that didn’t belong to them (Cloud
> Innovation) but rather belonged to Seacom through their acquisition of
> MacroLAN.
>
> While we regret this unfortunate incident, Mark’s claims that it was
> criminal or bad netizenship on the part of Cloud Innovation is without
> foundation and utterly incorrect.
>
> As shown below in the attached document from Paul Wollner(Ex-CTO of
> Macrolan who created IRR routes to allow Macrolan to announce Cloud
> Innovation's prefix); letter from Link Infinity International Ltd. (Link
> Infinity), A customer of Cloud Innovation; and attached LOA from LARUS
> authorizing IPDC Solutions to announce the prefix with origin AS134190.
> And a Letter from IPDC. This was an innocent mistake committed by third
> parties and had nothing to do with any action by Cloud Innovation or LARUS.
>
> Here’s what happened:
>
> Cloud Innovation delegated a /24 to Link Infinity, an ISP in December 2019.
>
>
> Link Infinity further delegated that same /24 to IPDC and asked Cloud
> innovation to issue an LOA, which we did. The LOA specifically required
> IPDC to use its own ASN to announce the space (AS134190).
>
> IPDC subsequently authorized one of its customers to use the said prefix.
>
>
> For reasons still unknown to Cloud Innovation, IPDC and their customer set
> up a BGP session wherein their customer used AS37353 as the origin to
> advertise prefix 156.241.3.0/24.
>
>
> Upon discovering the announcement, rather than contact Cloud Innovation,
> Mark contacted IPDC who provided him with an incomplete explanation blaming
> their customer and Mark, not realizing that Cloud Innovation was not the
> customer in question posted far and wide about the event. It is unclear to
> us why he chose to do this rather than contact us to try and resolve the
> issue.
>
>
> A contributing factor to the erroneous BGP configuration by IPDC's
> customer may have been data contained in some outdated IRR route objects
> for 156.241.0.0/16 which have subsequently been deleted.
>
> As soon as we became aware of the problem (via Mark’s email), we began to
> investigate the situation. As soon as it was clear that this was the result
> of third-party actions, we reached out to Mark privately to let him know
> what we knew and that we were still investigating. We delayed making a
> public statement in order to try and ascertain all of the facts of the
> situation. We prefer not to make public statements based on incomplete
> information.
>
> We apologize to the community for our small part in this unfortunate
> incident and assure you that we work very hard to remain good netizens and
> will address any concerns promptly when they come to our attention.
>
>
> Sincerely,
>
> Lu Heng
> CEO
> Cloud Innovations
>
> Attached:
> 1. Letter from Paul Wollner
> 2. Letter from Link Infinity
> 3. LOA Issued to IPDC Solutions for announcing 156.241.3.0/24 from
> AS134190
> 4. Letter from IPDC
>
> FYI: LARUS is proving IP management service for Cloud Innovation, while
> LARUS is also providing IP management service to other third parties in all
> regions, for full disclosure, LARUS and Cloud Innovation are headed by same
> CEO.
>
> Content of those letters have been posted here for your convince:
>
> *IPDC:*
>
> FOR IMMEDIATE RELEASE [Perusal of Cloud Innovation Ltd]
>
>
> IPDC Solutions Pte Ltd (IPDC) is a customer of Cloud Innovation Ltd and
> over the course of our corporate relationship we were given the authority
> to use address block 156.241.3.0/24 since 9th December 2019.
>
>
> On 12th December 2019, we have delegated that address block to our client.
> Following which our client further instructed us to announce the prefix
> with AS37353. In good will after minimal verification through WHOIS’ IP
> Prefix we have proceeded to execute our client’s request.
>
>
> On 7th May 2020 IPDC was then contacted by SEACOM, the legitimate holder
> of record for that ASN and have since learned that the customer’s use of
> that ASN was erroneous and not permitted by SEACOM and immediate action has
> been taken to rectify this situation.
>
>
> IPDC would like to apologize for our lack of attention in conducting
> thorough verification and wish to highlight that the involvement of Cloud
> Innovation Ltd in the entire process was providing the addresses to us and
> that we truly apologize as we understand that this incident may have
> indirectly implicated Cloud Innovation Ltd.
>
>
> IPDC however, does not wish to disclose our client information and our
> client information shall remain confidential under present circumstances.
> Once again, we apologize for our shortcomings.
>
>
> *Link Infinity:*
>
>
> To whom it may concern,
>
> We at HK Infinity International Ltd are a customer of Cloud Innovation and
> in the course received rights to use 156.241.3.0/24 from them. Beginning
> December, 2019, we delegated the right to announce this prefix to IPDC
> Solutions Pte Ltd. (IPDC). We asked Cloud Innovation to provide an LOA
> authorizing them to announce the space which was subsequently received.
> IPDC subsequently and without our knowledge delegated this space to one of
> their customers and allowed them to originate it from AS37353.
>
> This announcement was not authorized by us, nor is it permitted by the LOA
> provided by Cloud Innovation.
>
> Unfortunately, we were not aware of the situation until after it had
> already been resolved.
>
> It was never our intent to violate the LOA or to allow the prefix to be
> announced from a hijacked ASN. We do not condone this and apologize
> sincerely to the community for what has happened here.
>
> Sincere Apologies,
>
>
> *Paul Wollner:*
>
>
> 8 May 2020
>
> TO WHOM IT MAY CONCERN
>
> In the light of the recent email on NAPAfrica mailing list, I would just
> like to clear the air.
>
>
> The IP route objects were created by myself for Cloud Innovation when they
> signed up as a client of Macrolan ( now SEACOM) as they didn't have their
> own AS.
>
> At the time I was working for Macrolan (now SEACOM). I left the employment
> of SEACOM in October 2019.
>
> It appears that when Cloud Innovation's contract with SEACOM came to an
> end, the route objects were never cleaned up.
>
> This occurred after I left SEACOM's employment. Since leaving, I have no
> access to these objects.
>
> Regards
>
> Paul Wollner
> 082-786-9776
>
>
>
> On Fri, 8 May 2020 at 22:10, Mark Tinka <mark.tinka at seacom.mu> wrote:
>
>> Hi all.
>>
>> I'm not one to b**ch & moan in public, but per subject, I could not let
>> this one slide.
>>
>> And if you get this from multiple mailing lists, apologies for that - I'm
>> just trying to make sure that this reaches as wide an audience as possible,
>> on the continent.
>>
>> SEACOM (AS37100) acquired MacroLan (AS37353) a couple of years ago.
>> MacroLan is now part of the SEACOM family, and while we are in the process
>> of integrating that network into AS37100, some existing services continue
>> to be delivered on AS37353 until that exercise is completed.
>>
>> One of the customers that AS37353 was providing services to was Cloud
>> Innovation, in Cape Town. From a routing perspective, because Cloud
>> Innovation had no AS number for this service, all of their IP address space
>> was being originated by AS37353, on their behalf.
>>
>> In December of 2019, AS37353 ceased providing services to Cloud
>> Innovation. That is 6 months ago.
>>
>> In recent days, it came to SEACOM's attention that some of Cloud
>> Innovation's IP address space was being originated by AS37353 -
>> specifically, 156.241.3.0/24 - even though we were sure that they were
>> no longer a customer of AS37353 since December of 2019. At first, we
>> thought this was a cached entry in a number of popular looking glasses, but
>> then every looking glass had the same entry, which made this an unlikely
>> bug.
>>
>> As of yesterday afternoon, see below what Telia's looking glass was
>> saying about this prefix:
>>
>> *****
>>
>> Path #1: Received by speaker 0
>> 4809 134190 37353
>> 2.255.249.42 (metric 2134) from 2.255.253.101 (80.91.242.40)
>> Origin incomplete, localpref 200, valid, internal, best,
>> group-best, import-candidate
>> Communities:
>>
>> 1299:431
>> (RPKI state Unknown)
>>
>> 1299:1000 1299:30000 1299:30600 23456:20413 45352:4500 45352:9204
>>
>> *****
>>
>> But when I run a traceroute from my house to that prefix, it clearly was
>> not ending up in Cape Town, where AS37353's main operation resides:
>>
>> *****
>>
>> MacBook-Pro-7:~ tinka$ traceroute -I 156.241.3.1
>> traceroute to 156.241.3.1 (156.241.3.1), 64 hops max, 72 byte packets
>> 1 172.16.0.254 (172.16.0.254) 14.824 ms 11.522 ms 3.525 ms
>> 2 xe-1-3-0-1064.er-01-jnb.za.seacomnet.com (105.22.37.13) 5.620 ms
>> 9.714 ms 9.887 ms
>> 3 ce-0-2-0-0.cr-02-jnb.za.seacomnet.com (105.16.28.2) 175.232 ms
>> 172.699 ms 175.896 ms
>> 4 xe-0-0-0-8.cr-02-cpt.za.seacomnet.com (105.16.9.182) 164.496 ms
>> 163.578 ms 163.546 ms
>> 5 105.16.14.153 (105.16.14.153) 169.812 ms 171.272 ms 177.115 ms
>> 6 xe-0-0-0-0.br-02-lhr.uk.seacomnet.com (105.16.34.253) 168.911 ms
>> 172.958 ms 165.165 ms
>> 7 82.112.115.169 (82.112.115.169) 172.700 ms 176.482 ms 174.375 ms
>> 8 ae-17.r05.londen12.uk.bb.gin.ntt.net (129.250.2.147) 672.099 ms
>> 613.617 ms 615.109 ms
>> 9 ae-2.r24.londen12.uk.bb.gin.ntt.net (129.250.4.244) 181.952 ms
>> 183.087 ms 187.302 ms
>> 10 ae-16.r20.frnkge13.de.bb.gin.ntt.net (129.250.3.13) 190.511 ms
>> 185.579 ms 187.058 ms
>> 11 ae-3.r20.sngpsi07.sg.bb.gin.ntt.net (129.250.4.17) 520.882 ms
>> 613.982 ms 615.275 ms
>> 12 ae-9.r24.tkokhk01.hk.bb.gin.ntt.net (129.250.7.67) 612.301 ms
>> 586.886 ms 436.711 ms
>> 13 ae-1.r03.tkokhk01.hk.bb.gin.ntt.net (129.250.6.98) 614.470 ms
>> 613.416 ms 614.281 ms
>> 14 ce-0-3-0-3.r03.tkokhk01.hk.ce.gin.ntt.net (203.131.241.126) 614.128
>> ms 613.661 ms 615.416 ms
>> 15 * * *
>> 16 * * *
>> 17 156.241.3.1 (156.241.3.1) 494.400 ms 410.180 ms *
>> MacBook-Pro-7:~ tinka$
>>
>> *****
>>
>> So we, then, realized that this was a fraudulent use of MacroLan's
>> AS37353, to which we had given no express permission.
>>
>> As luck would have it, due to my days living and working in Malaysia, I
>> know the good folk that operate AS134190 (IPDC Solutions), who was the
>> upstream providing transit for this prefix. So I rang them up yesterday
>> afternoon, told them what was happening, and within the hour, they got that
>> eBGP session shutdown. I am most grateful to them for their quick response
>> and immediate understanding of what was going on. Even with all the
>> technology we have today, it, many times, comes down to having good friends
>> in good places.
>>
>> Anyway, it turns out the ISP that had acquired this prefix from Cloud
>> Innovation is based in Manila, Philippines. When IPDC delivered their
>> transit service to them in Manila, that ISP informed them that they should
>> use AS37353 for the eBGP session. Yes, one could argue that IPDC should
>> have done their checks to ensure that the AS number being provided by their
>> customer belongs to them, but to be honest, I'm not too bothered about that
>> compared to Cloud Innovation's thinking that it was okay to use another
>> network's AS number in order to deliver their services to their customers.
>>
>> IPDC confirm that this service was activated for the Manila ISP in
>> December of 2019, right around the time Cloud Innovation's service with
>> SEACOM, in Cape Town, ended.
>>
>> As it currently stands, the ISP in Manila is now off the Internet, with
>> some 12 paying customers currently without service. Their excuse - they do
>> not have an AS number of their own.
>>
>> IPDC tried to find out why the ISP in Manila thought that it was okay to
>> use another network's AS number for their service, and as it turns out, the
>> network engineer at the Manila ISP that set this up has since left the
>> company. All the ones currently there do not have any history about all of
>> this.
>>
>> Currently, 156.241.3.0/24 is not in the global BGP table:
>>
>> *****
>>
>> lg-01-ams.nl>sh ip bgp 156.241.3.0/24
>> % Network not in table
>> lg-01-ams.nl>
>>
>> lg-01-nbo.ke>sh ip bgp 156.241.3.0/24
>> % Network not in table
>> lg-01-nbo.ke>
>>
>> lg-01-cpt.za>sh ip bgp 156.241.3.0/24
>> % Network not in table
>> lg-01-cpt.za>
>>
>> *****
>>
>> That Cloud Innovation thought it was okay for them to use MacroLan's AS
>> number to originate their own prefixes into the global BGP is as morally
>> reprehensible as it is technologically distasteful.
>>
>> SEACOM have been working very closely with AFRINIC to delete previous
>> route objects from their IRR that certify a relationship between Cloud
>> Innovation's parent /16 aggregates that cover this prefix, and AS37353, but
>> this is one of those objects that cannot be removed via the MyAFRINIC
>> portal, and requires manual intervention from AFRINIC.
>>
>> I have not, personally, spoken to the proprietors of Cloud Innovation
>> and/or Outside Heaven, as I don't see how anything could explain this with
>> any degree of justification.
>>
>> For now, I will find some beer to wipe the foul taste from my mouth,
>> while we (SEACOM) consider what to do about this.
>>
>> And yes, for those who are wondering, RPKI's ROV would not have prevented
>> this, in its current form. This is AS hijacking, and ROV, today, tries to
>> solve the prefix-hijacking problem, first.
>>
>> Thank you for your attention.
>>
>> Mark.
>> _______________________________________________
>> RPD mailing list
>> RPD at afrinic.net
>> https://lists.afrinic.net/mailman/listinfo/rpd
>>
>
>
> --
> --
> Kind regards.
> Lu
>
> _______________________________________________
> RPD mailing list
> RPD at afrinic.net
> https://lists.afrinic.net/mailman/listinfo/rpd
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.afrinic.net/pipermail/rpd/attachments/20200525/9e76e35e/attachment-0001.html>
More information about the RPD
mailing list