Smyte at Carousell

2021-12-09

I just read about Twitter’s acquisition of Quill and how Quill would be shutting down 4 days later. This acquisition and the short notice period reminded me of some of Twitter’s prior acquisitions. In 2020 they acquired Squad and shut it down a day later. In 2018 they acquired Smyte (HN discussion) and shut it down with no warning.

In 2018 I was working at Carousell on the Trust & Safety team. We had integrated with Smyte for a good chunk of our fraud detection capability. In fact I was pretty happy with the functionality provided by Smyte at the time. Something I think they did well was to allow non-technical folk to update, test, and deploy new rules for detecting bad actors. The had good support as well; we had a shared Slack channel with some of their support personnel and their founders.

One Thursday evening on my way home I got a message that Smyte had just gone dark. Their APIs started returning failures, their support team and founders left the shared Slack channel, and Smyte was unreachable. We only found out what happened via Twitter’s public announcement.

The next day I arrived at the office early to discuss our next steps. The primary focus at this point was mitigation. We needed to get our fraud detection back up. Thus I began work on what we called the trust service. Luckily I had some dumps of the rules we used in Smyte, so I set about building a system that could replicate them.

By Saturday evening I had replicated some of our core rules, and the following week we deployed it to production. For a few days we put the system in a dry-run mode where we could monitor the decisions it made to measure its false positive rate. By the end of week we promoted the rules into the active mode, and the system started to take actions on bad actors.

Over the next couple months we enhanced the system to bring it closer to feature parity with what Smyte used to provide. At the same time we were looking into alternative third-party fraud detection providers.

Eventually we settled on Sift. Interestingly we passed on Sift initially when evaluating them next to Smyte.

Our experience with Smyte had burned us somewhat. Even though we use Sift for fraud detection, we’re also continuing to invest in extending the capabilities of our own trust service.

If there’s one learning from this whole Smyte fiasco, it’s to have a business continuity plan for the critical pieces in your infrastructure. This could be your fraud detection platform, or perhaps your 2FA provider. Twitter’s acquisitions show how your third-party integrations can vanish virtually overnight.