Nearly two years after the infamous July 8th Rogers outage in 2022, an independent review conducted for the Canadian Radio-television and Telecommunications Commission (CRTC) blames human error. The review says the outage was worsened by management and system “deficiencies.”

The CRTC commissioned Xona Partners in September 2023 to review the outage, a summary of which is now available on the CRTC website. Xona also looked at whether Rogers took sufficient steps to prevent another outage. The CRTC plans to release the full report at an unspecified date once it has redacted sensitive information.

Xona’s report summary details how Rogers was working on a seven-phase process to upgrade its IP core network — the outage occurred during the sixth phase of the upgrade process. According to the summary, an error in configuring distribution routers, which involved Rogers staff removing the Access Control List policy filter, caused the core network to be flooded with IP routing information that ultimately crashed the core network.

Staff removed the policy filter in an effort to “clean up” files in the distribution routers, and the “change management process, which includes audits of change parameters, failed to flag the erroneous configuration change.”

Also of note, the review summary says that Rogers’ core network hadn’t been configured with an overload limit, which meant there was no protection mechanism for the core network beyond the policy filter.

Moreover, the review summary notes that Rogers assessed the risk of the network upgrade as ‘high,’ but the company’s risk assessment algorithm downgraded the risk level following the successful completion of the first five phases. The ‘low’ risk rating meant staff weren’t required to conduct additional scrutiny for the upgrades, like conducting lab testing for the change or seeking higher levels of approval.

“The July 2022 outage was not the result of a design flaw in the Rogers core network architecture. However, with both the wireless and wireline networks sharing a common IP core network, the scope of the outage was extreme in that it resulted in a catastrophic loss of all services,” said the review summary. Additionally, it pointed out that it’s common for service providers to share an IP core network between wireless and wireline services to “balance cost with performance.”

Elsewhere, the review summary looked at factors that impacted network restoration. These include that Rogers’ management network relied on the IP core network, resulting in remote employees being unable to access the management network. “Rogers did not provision its network operation centre and other critical remote infrastructure sites with redundant connectivity from alternative service providers for network management.”

In a similar vein, the review pointed out that Rogers staff relied on the company’s network for their mobile and internet services, so when both networks failed, staff weren’t able to communicate effectively.

Finally, the review said Rogers staff “did not initially have access to the error logs from the failed routers and could not pinpoint the root cause for about 14 hours from the start of the outage.”

All of these factors prolonged the duration of the outage.

The review summary highlighted various measures Rogers took following the July 2022 outage to address deficiencies and improve network resiliency. The company added safeguards to routers in the core network to prevent another overload and also implemented a separate physical and logical management network. Rogers also deployed backup connectivity from third-party service providers to its network operation centre and other critical remote sites.

Other changes Rogers announced include separating its IP core network for wireless and wireline. However, this change remains a “work in progress.”

Moreover, Rogers switched to a new risk assessment algorithm, implemented organizational changes to improve collaboration between network operations and engineering teams, added an enhanced process for introducing new equipment and technology, and more.

Those interested can find the full review summary on the CRTC website. Source: CRTC Via: CBC News.

Quebecor has announced that it has successfully met all nine commitments related to its acquisition of Freedom Mobile. The company submitted a detailed progress report to Innovation, Science and Economic Development Canada (ISED) on July 3rd, outlining its adherence to the acquisition requirements.

One of the key commitments highlighted by Quebecor was the Mobility Price Freeze Guarantee, which ensured that wireless prices remained constant for all current and future Freedom customers. In addition to this, the company extended Freedom Mobile and Fizz services to new markets such as Manitoba and introduced low-cost 5G plans.

Quebecor emphasized its efforts to promote competition and lower wireless prices by offering affordable wireless packages. The company stated that these initiatives were instrumental in the significant 26.6% reduction in the wireless component of the Consumer Price Index in the year following the acquisition of Freedom Mobile.

The comprehensive progress report also addressed commitments related to preserving jobs, making investments, and harmonizing mobile plans. Quebecor's commitment to fulfilling these obligations demonstrates its dedication to enhancing the wireless market and providing value to customers.

For more information on Quebecor's fulfillment of the Freedom Mobile acquisition commitments, the full report can be found on their website.

Fido, the flanker brand of Rogers, has announced that its $34/20GB 4G plan will be coming to an end on July 8th. This news gives potential customers a few more days to take advantage of this offer before it disappears.

While the $34/20GB plan may seem like a good deal, there are better options available from competitors such as Freedom Mobile and Public Mobile. These providers offer $34/50GB 5G plans that also include Canada/U.S. calling, texting, and data usage. It is important to note that Fido's plan actually costs $39 per month but drops down to $34 if customers opt for automatic payments.

It remains uncertain what will happen to the 20GB plan once July 8th rolls around. Fido could decide to remove the plan entirely or adjust the pricing. In light of Virgin Plus recently increasing the price of some of its plans, including their 20GB plan jumping from $34 to $44, it wouldn't be surprising if Fido followed suit. However, there has been no indication that Fido's other plans will be affected by this change.

For those interested in exploring Fido's current plans and offerings, they can visit their website for more information.

Overall, with the deadline approaching, potential customers who are interested in Fido's $34/20GB plan should consider signing up before July 8th to take advantage of this offer before it potentially changes.