1 Online Hate and Violence: A Case Analysis of Incels and the Alt-right By Jalal Younes Under the supervision of Dr. Harpreet Aulakh An Honours Project submitted in Partial fulfilment of the Degree requirements for the degree of Bachelor of Arts - Criminal Justice Honours Mount Royal University April 27, 2023 2 Copyright 2023 Jalal Younes ALL RIGHTS RESERVED This work is completed in entirety by Jalal Younes. All rights are reserved to the information provided within this document MOUNT ROYAL UNIVERSITY CALGARY, AB, CANADA Abstract 3 Due to the increase in hate-motivated mass killing attacks being associated with the perpetrator’s involvement in online hate communities, concern has been growing in the media regarding the extent to which an individual’s online activity can radicalise them. These attacks can be motivated by a bias against race, gender, or religion, such as the 2021 Buffalo, New York shooting, where the shooter targeted the black community. They can also be gender-based, like the incel van attack in Toronto in 2018. This research project examines the connection between hate-motivated violence and social media. Using case examples of the Toronto Van Attack and the Tops supermarket shooting and a semi-systematic literature review, the findings suggest that social media, through design features, policies, memes, sense of belonging, and trolling, can influence individuals within these groups towards violence. Keywords: Online Hate, Involuntary Celibates, 4Chan, Alt-right, Social Media, Payton Gendron, Alek Minassian 4 Acknowledgements First, I would like to acknowledge my supervisor on this project, Dr. Harpreet Aulakh. She has shown extreme support, patience, and understanding throughout the process of this project and I am very thankful for having her as my supervisor and as a professor in previous years. Her course on diversity issues in the justice system is what inspired me to choose a topic of this nature for my honours thesis. Second, I would like to acknowledge my mother, M.Y. She has always been the most supportive and resilient person I have ever known and I am extremely grateful to have someone like her in my life. Table of Contents 5 Introduction....................................................................................................................................6 Methodology................................................................................................................................... 7 Literature Review.......................................................................................................................... 8 Social Media Design’s Influence on Toxic Community Formation........................................ 10 Reddit................................................................................................................................. 11 TikTok................................................................................................................................ 14 Online Hate Speech and Trolling.............................................................................................15 Defining Hate Speech........................................................................................................ 15 Trolling...............................................................................................................................17 Incels........................................................................................................................................ 18 Alt-right, 4Chan and 8Chan.....................................................................................................23 Case analysis.................................................................................................................................30 Alek Minassian........................................................................................................................ 30 Payton Gendron....................................................................................................................... 35 Conclusion.................................................................................................................................... 41 References..................................................................................................................................... 44 6 Introduction The Toronto Van attack was the deadliest mass killing in Toronto’s history, and it started when a man rented a van on April 23, 2018 and at 1:24 P.M., he drove that van over the curb and into pedestrians (Boyko, 2021). Ten people were killed and an additional 16 were injured as a result of this attack (Boyko, 2021). In Christchurch, New Zealand on March 15 2019, a gunman entered a mosque and opened fire on worshippers, killing 44 people. He then drove to another mosque and opened fire on more worshippers, killing an additional seven people. On March 14, 2022, an 18-year-old man walked into a supermarket in Buffalo, New York and opened fire on shoppers inside, killing 10 people and wounding three others (Thompson et al., 2022). These three attacks took place in three different countries, at different times, and the perpetrators targeted different groups of people. However, there is a common theme that present in these attacks and they represent a larger and novel trend when it comes to attacks of this nature. The perpetrators in each of these cases had an extensive online presence, where they interacted with like-minded people, shared ideas, memes, comments, videos, and manifestos. The websites that the perpetrators used include 4Chan, 8Chan, and Reddit. Researchers have identified several ways in which online hate communities can promote violent acts both in incel communities and alt-right communities. Regehr (2020) outlines a five-step indoctrination process which converts lonely men into angry incels online. She argues that, ultimately, the deification and martyrdom of incel killers in these communities promotes a cyclical process that encourages taking violent action. In the case of the alt-right, another researcher indicates that memes are used to spread alt-right conspiracy theories on the website 4Chan which results in a sense of urgency that encourages people within these groups to take violent action to “defend” their nation (Thorleifsson, 2021). The goal of the current research is to answer the research 7 question: “How do social media sites provide a venue for hateful ideas to translate into physical violence?” This research project explores the relationship between online hate and real-life hate crimes. Methodology The goal of this thesis is to understand the process which allows for an individual's hateful ideas to translate into real-life violence. A qualitative study was undertaken to provide an in-depth analysis of this phenomenon. Due to the dearth of research on the topic, an exploratory research design using a semi-systematic literature review and case study approach was used to explore the relationship between hate communities on social media and their influence on violent acts. The first case that was used in the analysis was the incel attack perpetrated by Alek Minassian in 2018. The second case used was the Tops supermarket shooting which took place in Buffalo, New York in 2022. These two cases were specifically chosen for two reasons. First, these two cases both presented a plethora of data that were used in the analysis. This included court reports and a police interview in the case of Alek Minassian and a manifesto in the case of Payton Gendron (the Tops supermarket shooter). These sources of data for the two cases were found using google search. Second, these cases represent two different types of hate-motivated crimes, gender-based and race-based. To collect relevant literature on the topic, a list of keywords was made. These keywords and phrases included the websites of interest such as 4Chan, Reddit, and 8Chan. Other keywords included references to the actual events such as the names of the perpetrators and the locations of the incidents, and direct references to the topic such as “online extremism” “incels'' and “incel violence.” Purposive sampling was chosen here as literature on the subject is scarce, as the topic 8 of interest represents a relatively new trend. Thus, acquiring as much relevant literature on the topic was important. The two cases chosen represent different types of extremism, as the perpetrators targeted different groups. In the case of the Toronto van attack, the perpetrator sought to target mainly women (although he also targeted men), making his attack a form of gendered violence. In contrast, the perpetrator of the Tops supermarket shooting in Buffalo, New York intentionally drove to a predominantly black neighbourhood to target black shoppers. This purposive sampling allows for the choosing of this diverse pair of cases and increases the validity of the results. Moreover, these cases were high-profile - which provided many sources of data such as manifesto’s, police interviews, and court reports. Choosing cases with a higher amount of data available provides a more comprehensive analysis and understanding of the process of how hate communities on social media influence individuals towards violence. Literature Review In recent years, several mass killings have been connected to the perpetrator’s affiliation and involvement in online hate communities such as 4Chan (alt-right) or incels. This is a recent phenomenon, as these groups have emerged due to the advent of the internet and social media. The literature which explores the connection between social media and violence revolves around four major areas, which are discussed in this section. The first area of discussion covers the way in which social media companies design their products. This is important because these design features can promote the emergence of hateful communities and the ideas that circulate within them, which is what will be discussed in the second part of the literature review. The third portion of the literature review covers hate speech and trolling within these communities. Finally, the literature review will discuss two hate communities specifically: incels and the alt-right. Social Media Design and the Attention Economy 9 Understanding the connection between social media and hate speech requires delving into the design of social media and their structure and algorithms. This information will help in answering the question: how do social media sites provide a venue for hate speech to translate into real-life violence. Bhargava & Velasquez (2021) explain that there exists a new and unique form of business that seeks revenue from an individual’s attention - and this “attention” is sold to other companies and buyers. This phenomenon has been widely referred to as the “attention economy” (Bhargava & Velasquez, 2021). In their research, the authors cover some of the design features that social media companies use to make their products more addictive. These design features include: intermittent variable rewards, social validation, and erasure of natural stopping cues. Bhargava and Velasquez (2021) explain that social media companies will sometimes design their website in a way that rewards users in varying magnitudes and frequencies. Second, social media is designed in a way that pressures the user to use the product through seeking validation of others. Finally, the researchers explain that social media sites now often feature an endless scrolling potential, which goes against how traditional websites worked - where scrolling to the end of a webpage would mark the end. Montag et al. (2019) cover some of the same and some additional major design aspects of social media that prolong app usage and make them “addicting;” These tactics include: endless scrolling/streaming, the endowment effect, social pressure, showing users what they like, social comparison and reward, and the Zeĭgarnik effect. The researchers explain that app and game designers want their users to be immersed in their application in a way that distorts the user’s sense of time when using the app. One way this is achieved is by using endless scrolling, where the content that a user is able to view or interact with is seemingly endless (i.e. YouTube 10 recommending another video after the end of a video, Montag et al., 2019). The second design strategy is the endowment effect, which the researchers describe as the increasingly difficult time that a user will have detaching from an app after spending an increasing amount of time on it. The third tactic is social pressure, which, as the name suggests, involves the pressure of other people that leads to usage of the app (Montag et al., 2019). Fourth, the researchers describe how social media apps are designed to show users what they want to see. The fifth design strategy is social reward and comparison, which, in simple terms, refers to the reward given by other users in the form of likes, for example (Montag et al., 2019). Lastly, there is the Zeĭgarnik effect, which is aptly explained by the researchers: In sum, these classic works from the field of psychology suggest that individuals involved in the execution of high investment tasks, react with (emotional) strain if interrupted. The final completion of the task will remove this strain. (p. 7) The researchers exemplify this effect by using the game “Candy Crush,” where players have to complete increasingly difficult levels. There are limited attempts that a player has in a certain span of time; due to the difficulty of the level, players exhaust their attempts and get frustrated, but are simultaneously attracted by the game and thus end up spending money for extra lives (Montag et al., 2019). These very design features, while they might seem unconnected to hate speech at first, can actually be part of the reason why hateful content and ideologies are promoted on certain websites. This is covered further in the next section. Social Media Design’s Influence on Toxic Community Formation The previous section explored how social media companies design their products in certain ways. This section will discuss how these design features can promote the emergence of 11 toxic/hateful communities and the circulation of hateful ideas within them. Two specific social media sites are covered, Reddit and TikTok. Reddit Many researchers have investigated the ways in which social media design features promote toxic communities. Adrienne Massanari (2016) elucidates this problem by studying two phenomena that occurred on Reddit. These are: The Fappening and Gamergate. Massanari (2016) explains that Reddit is a community of communities, and many of the communities on Reddit revolve around “geek masculinity.” “Geek interests” on Reddit primarily concern technology, science, computers, world news, and popular culture (Massanari, 2016). Spaces dedicated to geek interests show a disposition to treat women as either objects of sexual interest or as unwelcome, and sometimes as both. Massanari (2016) explains that Gamergate was an online phenomenon that originated on Reddit which started a campaign dedicated to delegitimize and harass women in the gaming community. The other case that Massanari (2016) describes is “The Fappening,” a community that briefly formed on Reddit which would post leaked nude images of celebrity women. This subreddit (a smaller community within the larger Reddit community) was not banned immediately by moderators until it was revealed that there were images of an underaged celebrity circulating (Massanari, 2016). Massanari (2016) suspects that the reason this subreddit was allowed to exist for as long as it did had to do with monetary incentive for Reddit. Massaanri believes that there are several designs and policies of Reddit that enable such “toxic technocultures” to exist. First, Reddit features a “karma” system, which is a number that is displayed on each user’s account that quantifies their contribution to the community (Massanari, 2016). This number increases when a user makes comments or posts that others on the site upvote (Massanari, 2016). This can become 12 problematic when, as exemplified earlier, users are rewarded for spreading nude images of celebrity women on the site (Massanari, 2016). There is also the fact that Reddit displays the most upvoted or “best” comments and posts at the top and requires individuals to manually change the filter to see more controversial material - this creates a “herding” effect which biases users to vote in a similar way as others (Massanari, 2016). Massanari (2016) also describes the aggregation of content across subreddits. Basically, there is a page on Reddit that displays content from across different subreddits, which can sometimes contain content from “objectionable” subreddits such as Gamergate and The Fappening (Massanari, 2016). The author argues that this problem is further compounded by the fact that new users who see this type of content that they do not like will avoid the site and thus their views will not be reflected. The author also explains that the platform’s policies allow for the existence of toxic technocultures, as Reddit takes a “neutral stance” on the content it hosts and only intervenes when one of the major site-wide policies are broken and offers an ineffective way for users to report objectionable subreddits (Massanari, 2016). Although the author focuses on Reddit in this study, Massanari (2016) emphasises that due to the fluidity and interconnectedness of different online platforms, these interactions on Reddit are “reflective of and influenced by other platform cultures” (p. 341). It is important to note that Reddit is a platform that has changed significantly over the past few years and thus it is important to acknowledge the limitations of this study as it applies to Reddit today. The policies of Reddit have changed even during the duration of Massanari’s (2016) study, which she acknowledges at the end of her paper. For example, she notes, some policies have been created to target the harassment of other users. However, the concept of geek 13 masculinity is something that persists on Reddit. Subreddits pertaining to science, technology, gaming and world news continue to be among the most popular on Reddit. In a more recent study, Gaudette et al. (2021) investigate how alt-right communities on Reddit use the upvote feature to develop a sense of collective identity. Gaudette et al. (2021) explain that online extremism can be understood through the lens of collective identity, which refers to the formation of a “we” among community members as well as an “us vs. them” binary (Gaudette et al., 2021). The researchers focus on one community in particular, “r/The_Donald,” a subreddit dedicated to discussion relating to the then-U.S. president Donald Trump (Gaudette et al., 2021). The rules of this subreddit specified that while racism and antisemitism were not permitted, muslims and illegal immigrants were not a race (Gaudette et al., 2021). This subreddit contained a plethora of discourse surrounding “white genocide,” anti-Islam, and anti-black topics (Gaudette et al., 2021). However, the researchers uncovered two dominant themes when analysing the most upvoted comments on this subreddit. These include external threats and internal threats. Gaudette et al. (2021) found that a significant portion of highly popular and highly upvoted comments revolved around the perceived threat of Muslim immigration to the United States. It was consistently found that highly-upvoted comments involved viewing Muslims as invaders of “the West” and a threat to white people (Gaudette et al., 2021). One interesting theme found among some of these comments by the authors was the “emerging sense of desperation that the survival of their in-group is being threatened by the Muslim out-group” (Gaudette et al., 2021, p. 3498). The authors also note that many members in these comments would often suggest, as the authors call it, a “defensive stance” against the perceived threat of Muslim immigration. However, it was rarely explained how one might defend the survival of the in-group, the authors suggest that these members may have avoided overtly condoning violence 14 as a solution in order to avoid having the subreddit banned by Reddit (Gaudette et al., 2021). The authors argue that the way the upvote system on Reddit was used by this community allowed for these comments to be highly popular and highly-visible compared to other comments that denounced racism within this community, which received less upvotes, if any at all. TikTok TikTok is a highly popular social media app which features a “for you page,” a page that you can endlessly scroll on and features relatively short videos. The idea of the for you page is that as you progressively interact with videos that you like by either liking the video or its comments, commenting on it, or sharing it, the sophisticated algorithm will then show you more similar videos on the for you page (Boucher, 2021). These reflect two major strategies used by social media that Montag et al. (2019) describe in their research: endless scrolling and showing you what you like, which are described in this chapter. Boucher (2021) conducted a unique auto-ethnographic experiment where he created a TikTok profile with “moderate conservative beliefs” and found that the algorithm slowly and progressively recommended increasingly radical content throughout the 1,000 videos he consumed. Some of the far-right themes that Boucher (2021) found included “extreme dissent, the great replacement, accelerationism, hypermasculinity, coded Nazi paraphilia or symbolism, misogyny, racism, and trans- or homophobia” (p. 58). Boucher (2021) found that users were able to avoid getting banned by TikTok by simply sharing “back-up accounts,” accounts that the user’s followers can follow in case the current one they are using gets banned. Other tactics included censoring videos and using coded or alternative words. Another observation that Boucher (2021) made is that the comments sections on relatively mild videos were more extreme in nature than the videos, suggesting that comments 15 were less monitored by TikTok than videos were. Some of the comments even contained calls for violence (Boucher et al., 2021). Boucher (2021) also explains his observations regarding belonging in these communities: Extreme content creators express and play upon a shared apparent need to connect. I have observed a need for community and for camaraderie on their behalf that translates into a quest for sense – the renegotiation of identity, and sacred values, also mentioned in the literature. (p. 65) Thus far in the literature review, I have talked about the structure of social media sites, their features and characteristics and their ultimate purpose, which is to keep users on for the longest possible time. I have also looked at literature which describes the ways in which these common design features can be conducive to the existence of misogynistic and alt-right communities. The next section describes the literature pertaining to hate speech and hate communities online and further narrowing the review to two specific online hate communities: Incels and the Alt-right The aim here is to understand the key ideas behind these communities, their demographic and how they lead individuals to commit acts of terror in real life. Online Hate Speech and Trolling In this section of the literature review, two things are covered. First, the definition of “hate speech” is discussed, along with the challenges that it poses. A theory that attempts to explain the motivation for online hate speech is briefly reviewed. Then, the concept of trolling and cyberhate are discussed within the context of online hate communities. Defining Hate Speech For the purposes of this study, hate speech has to be defined. However, this is not a simple task as hate speech can have varying definitions based on its intended function. A legal 16 definition can not be used here as different nations have different laws and definitions for hate speech. An academic definition is used instead. According to Anderson & Barnes (2022), there are four definitional bases for hate speech: harm, content, intrinsic properties and dignity. Harm-based definitions of hate speech attempt to identify elements of speech that subject a group to certain disadvantages, which includes things such as linguistic violence and discrimination (Anderson & Barnes, 2022). Content-based definitions aim to identify speech that expresses hatred against a group of people who can be identified by features such as race, religion, sexual orientation, etc.(Anderson & Barnes, 2022). Intrinsic properties definitions focus on the actual content of the speech communicated to identify the presence of speech that is inherently derogatory such as racial slurs (Luvell and Barnes, 2022). The final definitional base identified by these scholars is dignity, which focuses on identifying speech that harms the dignity of its targets. The current study uses Zhang & Lou’s (2018) definition of online hate speech: Any communication that disparages a person or a group on the basis of some characteristics (to be referred to as types of hate or hate classes) such as race, colour, ethnicity, gender, sexual orientation, nationality, religion, or other characteristics. (p.1) This definition coincides with two of the definitional bases described above: Content-based and dignity. Various researchers have attempted to theorise online hate speech and its motivations, and many explanations centre around anonymity. Kilvington (2021) attempts to create a theoretical framework based on the work of Goffman (1922-1982). In simple terms, Goffman theorises that human interaction is split into frontstage and backstage performances (Kilvington, 2021). The frontstage consists of a “performance” of expected behaviour and an anticipated image to an audience (Kilvington, 2021). The backstage is the opposite, it is a private world with 17 no viewers where individuals engage in behaviours that they would not engage in on the frontstage, this includes virulent views (Kilvington, 2021). What this author suggests is that due to four factors: anonymity, invisibility, dissociative imagination and rapid response, individuals are disinhibited and thus more likely to engage in online hate. Kilvington (2021) argues that the online space becomes a place where the boundaries between the frontstage and backstage are blurred, and thus individuals are more likely to communicate virulent ideologies and attitudes that would otherwise be reserved for the backstage, where no audience exists. In this section, the term hate speech was defined, as it relates directly to the research question. Furthermore, a theory that attempts to explain the motivations of online hate speech was discussed, which provided insight into some of the factors that makes online hate speech unique. In the next section, the concept of trolling will be discussed in the context of online hate communities. Trolling To understand online hate communities, it is important to understand what trolling is. Trolling is a blanket term that refers to a type of negative behaviour that people engage in online - from basic provocative behaviour to racism and misogyny (Golf-Papez & Veer, 2017). Because this behaviour is unique to the online space and has connections to online hate communities (Hodge & Hallgrimsdottir, 2019), looking at the relevant literature helps in understanding the link between online platforms and online hate. In her analysis of alt-right trolling, DeCook (2020) describes trolling as a “political aesthetic.” DeCook (2020) argues that the way in which members of the alt-right engage in the politics of “being as outrageous as possible” allows for members to hide behind the guise of “joking.” Using trolling or “joking” as a cover for saying or doing outrageous things absolves the 18 alt-right of blame when they are faced with the consequences of their actions (DeCook, 2020). DeCook (2020). A real example of a trolling campaign started by the alt-right is the co-opting of the “ok” hand symbol (DeCook, 2020). In 2017, white supremacists on 4Chan started the hoax by claiming that the “ok” hand symbol was a white supremacist symbol. This was done because this group on 4Chan wanted to show that the media was gullible enough to fall for this hoax (DeCook, 2020). Although the anti-defamation league originally emphasised the origin of the hoax and did not list it as a hate symbol, that was later changed due to the widespread use of the symbol by white supremacists to signal belonging to the group (DeCook, 2020). DeCook (2020) argues that this is an example of trolling as a political aesthetic, as what had started as a hoax to create confusion, had later become an actual symbol of white supremacy. The scholarly literature on the topic highlights two main findings: first, the structure and design of social media can lead to the propagation of toxic ideas and groups. Second, the concept of hate speech and how trolling can be utilised by hate groups to their advantage. The next section delves into detail about two specific groups, their origin, their beliefs, and how these groups produce individuals who commit acts of violence. These two groups are incels and the alt-right. Incels The term “incel” is a portmanteau of the phrase “involuntary celibate,” which, as the label suggests, refers to individuals and their struggle with forming romantic and sexual relationships (Sparks at al., 2022). The term had an early origin, dating back to 1997 (Spark et al., 2022); Alana, a sexually frustrated woman created a website wherein she documented her struggles in dating (Sparks et al., 2022). Soon, the site functioned as a support group for other people sharing these struggles, featuring a discussion forum and other forms of communication 19 between members (Sparks et al., 2022). The original group of people who identified as incels were diverse, sharing multiple genders and sexual orientations (Sparks et al., 2022). However, soon after Elliot Rodgers' incel attack, Alana discovered that the community had transformed into a male-only group that primarily blames women for their frustrations (Sparks et al., 2022). Elliot Rodger is a mass murderer who went on a stabbing and shooting spree in Isla Vista, California, causing the death of six people, and he is an important figure when discussing incels due to how the community idolises him (“Elliot Rodger,” 2018). Rodger had posted videos on YouTube for a while before his attack, with some videos espousing his frustration with why women did not want to have sex with him (“Elliot Rodger,” 2018). In the manifesto/autobiography that Rodger emailed (which has now been shared everywhere) he details how he plans to get revenge on society, which he claims has denied him love and sex (“Elliot Rodger,” 2018). Rodger specifically talked about members of the Alpha Pi sorority at his college campus, which he planned to target in his attack (“Elliot Rodger,” 2018). This attack is widely regarded as the “first” incel attack (although some scholars debate this), and Rodger is idolised in incel communities (“Elliot Rodger,” 2018). Researchers have identified several key concepts or tenets that underpin the incel “ideology.” One of these tenets is the “redpill” and the “blackpill;” the former is a reference to the film “The Matrix,” and “taking the redpill” refers to the state of becoming aware of the truth about the relationship between men and women - a view that portrays women as vain and vacuous (Stijelja & Mishara, 2022). Another characteristic view of the redpill is the idea that women hold power over men in modern society (Lindsay, 2022). Some of those who have “taken the redpill” according to some researchers, have traditionally resorted to misogynistic dating advice from a community called “Pick-up artists” (Stijelja & Mishara, 2022). However, this 20 contrasts with the blackpill, which is rooted in a similar view. The “blackpill” is a state of total hopelessness that many incels purport experiencing, wherein the idea of having sexual relations is completely unachievable; this comes from the view that women will only initiate these types of relationships with men who fit certain criteria, many of which are related to physical uscattractiveness (Lindsay, 2022; Stijelja & Mishara, 2022). Other traits include neurotypicality, race, personality, and muscularity, which incels believe are traits that are unchangeable and contribute to what they perceive as society that is not only stacked against them, but oppresses them (Lindsay, 2022). The blackpill is vital to understand the motives behind incel terrorism; Lindsay (2022) further explains: It was found that the black pill simplistically categorises people into a three-tiered system. This hierarchy is primarily based on physical attraction: a minority of alpha males and ‘desirable’ females are located at the top, a majority of ‘average-looking’ ‘betas’ or ‘normies’ follow, and a minority of physically ‘unpleasant’ (exclusively male) incels are found at the bottom (Baele, Brace and Coan 2019; Lindsay 2020). The black pill purports that such structural hierarchies are impermeable and that transcending one’s genetics is impossible. (p. 215) The view that certain structural hierarchies are impenetrable is a core belief in extremist worldviews (Baele et al., 2019, as cited in Lindsay, 2022), and Lindsay (2022) suggests that this worldview is what influenced terroristic attacks by incels, who perceive their actions to be an attempt at retribution on those who hold a higher social status. Other researchers have also made a connection between incel ideology and extremism/violence. O’Mally et al. (2020) suggest that there are five “interrelated normative orders” which include: “the sexual market, women as naturally evil, legitimising masculinity, male oppression, and violence.” The first four normative 21 orders, the researchers argue, play into the legitimisation of violence by incels mainly against women. More specifically, the researchers explain how the idea of “sexual market value” (SMV) feeds into the feelings of injustice, which in turn legitimises formal and informal forms of violence against women (O’Mally et al., 2022). Williams et al. (2021) analysed the oral and written communications of seven known violent incels and found common themes relating to revenge, power and hate which are similar to other violent mass offenders. Another group of researchers performed a diachronic analysis of language on multiple incel platforms, and found that although variations exist, violent and dehumanising language had increased steadily from the period spanning 2016-2022 (Baele et al, 2023). There is an argument to be made, based on these data, that the incel ideology itself is extremist in nature. Simply being exposed to such ideologies on the internet may play a role in adopting these views, as O’Mally, Holt & Holt (2022) explain. Although the incel ideology appears to be extremist in nature, that does not explain why some men become incels and others do not. Understanding the characteristics of the incel demographic can help explain the link between social media and real-life violence. Kaitlyn Regehr (2020) argues that contrary to the popular narrative that incel terrorism is the result of “lone-wolf” actors or “rare evil individuals,” incels actually face a pattern of indoctrination. Regehr (2020) suggests a 5-step process in which, she argues, individuals get indoctrinated into the incel community. The first step of the indoctrination process involves individuals who are susceptible to indoctrination beginning to seek ways to reduce or cope with their loneliness (Regehr, 2020). Regehr’s (2020) research reveals that several incels deal with experiences of social rejection, self-esteem issues and anxiety. In fact, after physical appearance, incels in this study chose “self-confidence and anxiety” as a factor that prevented them from 22 finding a romantic partner, with 74.2% of respondents choosing this factor. A significant portion of the respondents indicated feelings or isolation and loneliness, which Regehr (2020) suggests led these individuals to finding belonging in incel communities online. The second step involves the transformation of loneliness into anger through the incel ideology - placing the blame on women and creating a misogynistic focus (Regehr, 2020). Regehr’s (2020) research reveals that many incels follow the concept of lookism, which is a term that describes what incels perceive to be discrimination based on physical appearance. This concept coincides with ideas of the “blackpill” discussed earlier. Regehr (2020) describes the third step as the “rich toxic tapestry culture,” which describes the incel community culture and their tendency to promote content creation as well as dark and edgy humour. It is through this culture, Regehr (2020) argues, that terminology such as “Chads” and “Staceys” are used in order to exemplify individuals perceived to have higher sexual and social power, and it is these individuals that incels seek to deride and destroy. Regehr (2020) also suggests that the internet’s “echo-chamber effect” solidifies these beliefs through the constant consumption of violent misogynistic and anti-feminist content, this is the fourth step of the indoctrination process. The final step in the indoctrination process involves the deification and martyrdom of individuals who have committed mass violence, Regehr (2020) explains: “there is cyclical nature to the violence, which is encouraged, documented, shared, celebrated and encouraged again” (p.140). There are several other studies that investigate the link between mental health and incels. Sparks et al. (2023) found that compared to non-incel men, incels experienced a higher degree of loneliness and a lower amount of social support. In addition, the findings show that incels are more likely to engage in unhealthy coping mechanisms such as self-blame and behavioural disengagement (Sparks et al., 2023). One of the interesting findings of this study pertain to the 23 use of “venting” as a coping strategy. Sparks et al. (2023) suggest that incels use incel communities as therapeutic spaces. In a survey study, Moskalenko et al. (2022) asked a sample of 274 incels to report on their self-diagnosed and officially diagnosed mental health issues. The findings revealed that 95% and 94% of the sample reported feelings of depression and anxiety, respectively. What the study also revealed is that 18% of the sample reported having a formal diagnosis of autism, and 74% reported having autism-like traits (Moskalenko et al., 2022). This number contrasts sharply with the estimated 3.6% of males in the general population who are formally diagnosed as autistic (Moskalenko et al., 2022). The authors also found that having a formal diagnosis of autism in addition to a history of being bullied correlated significantly with radicalism but not with ideology. As evident from the research above, mental health issues and neurodevelopmental disorders such as autism are prevalent among the incel community. Alt-right, 4Chan and 8Chan The alt-right is the other group of interest in the present study. 4Chan is a website that has become a notorious hub for white supremacy. This website, along with similar sites like 8Chan, have been linked to several acts of terrorism such as the Christchurch shooting. This section of the literature review will first go over what 4Chan is and how the site actually works. The rest of this section will cover the hate-speech aspect, going in detail over how the alt-right community operates on that site and similar sites to uncover the major themes. Ultimately, these themes have been used for analysis of specific cases in the next section of this study. 4chan, as described on their website, is a “simple image-based bulletin board where anyone can post comments and share images” (4Chan, 2023). The FAQ section describes the basic aspects of the site and how users can use it. Users can start what are called “threads,” by 24 first positing at least an image, optionally accompanied by text (4Chan, 2023). Then, other users can reply to that thread by posting either an image, text, or both (4Chan, 2023). Making an account is not required to participate on this site, making this site largely anonymous (4Chan, 2023). Moreover, there are different boards for different topics such as science, technology, sports, business, fitness, cooking, pornography, and politics. The most virulent political image board on 4Chan is called “politically incorrect,” which is where a lot of the alt-right content is found. A similar site, called 8Chan, 8Kun, or Infinity Chan, was designed very similarly to 4Chan, but placed even more emphasis on free speech and anonymity (Wendling, 2019). It was started in 2013 because the creator was frustrated with 4Chan (Wendling, 2019). Since then, especially in 2019, the site has been associated with three mass shootings (Mezzofiore & O’Sullivan, 2019). The first is the Christchurch shooting - followed by two other attacks by John Earnest and Patrick Crucius, both of which cited the Christchurch attack inspiration (Mezzofiore & O’Sullivan., 2019). In August of 2019, the site was overrun by cyber attacks and eventually shut down due to a web service provider ceasing security services to the site (Perryer, 2020). Thus, the site is no longer up and running, but 4Chan remains accessible to anyone on the internet. Thorleifsson (2021) argues that the way in which cyber fascism is produced on 4Chan’s politically incorrect board amplifies the belief of an ultra-nation under attack and in need of urgent violent defence. This belief is rooted in what is called the “Great Replacement Theory,” and it is vital to understand what this conspiracy theory is to understand how it leads some people to commit acts of violence (Thorleifsson, 2021). 25 The Great Replacement Theory, otherwise known as white genocide, is an alt-right conspiracy theory, and thus follows some of the core beliefs of the alt-right (Davis, 2022). The three main interconnected beliefs of the alt-right ideology are white supremacism, antisemitism, and hegemonic masculinity (Davis, 2022). Alt-right ideology is biologically focused, which can be exemplified by terms like “human biodiversity” and “race realism” (Davis, 2022). Davis (2022) also explains that the alt-right, although rooted in white supremacism, is not concerned with biological hierarchy but rather with the idea of biodiversity, which is where terms like “white nationalism” and “white separatism” come from (Davis, 2022). Thus, there is a belief that races should have their own ethnic homelands, and the goal becomes the establishment of “pure white racial states” in countries that were formerly white (Davis, p. 4, 2022). The idea of white supremacism also has another interconnected dimension, and that is the belief that the white race is subject to an orchestrated attack that aims to eliminate it (Davis, 2022). That effectively sums up the first of the three components, which is white supremacy (Davis, 2022). The second component is antisemitism. Davis (2022) articulates that, to the alt-right, the Jews are seen as leading the white race down a path of “degeneracy,” and are believed to be behind many things wrong with society. Jews are seen as being the ones who are orchestrating the elimination of the white race (Davis, 2022). The alt-right also attributes other “societal ills” to the Jews, these include liberalism, feminism, globalism, immigration, multiculturalism, and diversity (Davis, 2022). These despised philosophies/practices are believed by the alt-right to be weaponized by the Jews to bring about the extermination of the white race (Davis, 2022). The third component of the alt-right is misogyny (Davis, 2022). Davis (2022) explains that masculinity is seen as being under threat due to the advancement of women and the propagation of homosexuality and non-traditional conceptions of gender identity. The alt-right views women who do not breed and 26 have children, as well as feminists who discourage reproduction, as propagating the aforementioned “white genocide” (Davis, 2022). Going back to Thorleiffson (2021), she explains that 4Chan’s inner workings are characterised by anonymity and a community that offers a sense of belonging. Another major characteristic of 4Chan is what Thorleifsson (2021) calls a “play frame of exploratory behaviour” where trolling and “shitposting” (posting derailing and provocative content) are prevalent. Thorleifsson (2021) argues that this “play frame” is a psychological state in which things are treated as serious and not serious at the same time, which lowers the barrier for participation for users. Another technical aspect of 4Chan in terms of design is that it rewards users for posting content that draws a lot of attention through replies (Thorleifsson, 2021). This is reminiscent of the discussion in the beginning of the current paper about the design of social media, namely, “social reward.” According to Thorleifsson (2021), cyber fascism on 4Chan relies heavily on memes to spread and reinforce myths about the threat posed by certain groups. Thorleifsson (2021) explains memes as: In the online realm, memes are considered to be ‘(a) a group of digital items sharing a common characteristic of content, form and/or stance; (b) that are created with awareness of each other; and (c) were circulated, imitated, and transformed via the Internet by many users’ (Shifman, 2014, p. 41) Memes, then, become a way for groups to reinforce the community bond and demarcate themselves from outsiders, who are deemed to be too “mainstream” (Thorleifsson, 2021). One of the major memes that circulate on 4Chan, according to Thorleifsson’s (2021) research is “Pepe the Frog,” a humanoid frog cartoon that is usually seen clad with Nazi attire and signifiers. 27 Another meme is the “Happy Merchant,” which is a drawn image of a Jewish man, with overly exaggerated and stereotyped features like a long hooked nose, crooked teeth, and a receding hairline (Thorleifsson, 2021). The portrayal shows the man “greedily” rubbing his hands together, and this image is frequently posted along with text describing how powerful Jews are in control of world affairs and are perpetrating a white genocide (Thorliefsson, 2021). According to Thorleifsson’s (2021) research, the two interconnected conspiracy theories that dominate the image board is white genocide and great replacement. As described earlier, the great replacement is a conspiracy theory that suggests the Jews are orchestrating an attack on white people using things like liberalism, multiculturalism, diversity, and feminism (Davis, 2022). However, on 4Chan, this theory has a more pronounced islamophobic dimension, going as far as suggesting that Jews are using Muslim immigrants as a sort of weapon to systematically replace white people in the west (Thorleifsson, 2021). The conspiracy theory of Eurabia, for example, suggests that European governments have decided, in secret, that Europe should be colonised by Muslims (Thorleifsson, 2021). As such, the prominent myth circulating on 4Chan is that Christian nations are under threat of Islamic invasion (Thorleifsson, 2021). One interesting detail about 4Chan is that those who believe in these conspiracies label themselves as “redpilled,” the same term that Incels use to describe their so-called awakening to the depressing nature of women and dating. Unsurprisingly, there is also a lot of misogynistic discourse on 4Chan, much of it relating to falling birth rates, and white women are blamed for embracing feminism and rejecting traditional gender roles (Thorleifsson, 2021). Adherents of this ideology reject any democratic means to further their cause, they see the democratic system as corrupt (Thorleifsson, 2021). Furthermore, the memes are spread on 4Chan perpetuate the logic of what Thorleifsson (2021) describes as the” ultra-nation/race is in 28 crisis and needs to be saved from its present state of disintegration through the agency of white men who are ready and prepared to fight to combat them” (p. 292). This is highly important, as this elucidates the logic that drives individuals from simply being “racist” online to feeling as though they need to take urgent action, which is terrorism. In her research on the 4Chan board, Thorleifsson (2021) found that what users believed they needed to do was to accelerate the collapse of modern society, which then will lead to a race war and, ultimately, white rebirth. What Thorleifsson described here is defined as accelerationism, a belief that using violence to collapse the current societal structure is needed to install new power structures that cater to the “needs” of white people (Institute for Strategic Dialogue, 2022). Thorleifsson (2021) found several themes when analysing the case of the Christchurch shooter. First, that he believed in the great replacement, and saw Islam as a threat to the white race and European culture. Second, that the shooter made several references to many earlier shooters some of which are Alexandre Bissonete, the Quebec mosque shooter, and Anders Breivik, the Norwegian terrorist who also perpetrated his attack on alt-right grounds. Even the magazines that Tarrant used for his weapon were covered in the names of people with anti-Muslim and anti-immigration views (Thorleifsson, 2021). Thorleifsson (2021) calls this the sanctification of terrorists. Shortly after Tarrant’s attack on the mosques in Christchurch, 4Chan users showered him with praise, made memes about him and portrayed him in an almost god-like manner (Thorleifsson, 2021). Many of the users also urged others to follow in his footsteps and commit similar atrocities, to which some did, like Patrick Crusius and John Earnest, who both cited Tarrant as their inspiration (Thorleifsson, 2021). One other observation that Thorleifsson (2021) makes is that Tarrant modelled his attack in a way that catered to the trolling and shitposting humour subculture on 4Chan described 29 earlier, meant to bewilder outsiders and entertain insiders. One example is how Tarrant put a camera on his helmet in order to simulate a video game perspective for the viewers of his livestream (Thorleifsson, 2021). Another example is how Tarrant comments throughout the video as if he is in a video game, and says completely out of context phrases like “subscribe to PewDiePie (Thorleifsson, 2021). PewDiePie is a very popular YouTube content creator with over 100 million subscribers. The literature review explored the complex factors and processes that are involved in the overall process in which individuals on social media are radicalised towards violence. The first section of the literature review discussed the concept of the attention economy and how social media companies design their products to cater to this economy. These very design features are shown to be problematic in some cases, where they have an effect of promoting the emergence of toxic communities and the circulation of hate speech within them. Next, the concept of hate speech was discussed and a definition was provided. A theory explaining online hate speech was briefly reviewed in this section as well. The concept of trolling was also introduced and the ways it is utilised by alt-right communities was explored. Finally, the literature review covers two specific online hate communities: incels and the alt-right. The findings show that there is a connection between social media and real life violence in two major ways. First, there is the structural aspect of social media. This includes design features as well as site policies, both of which are shown to promote toxic communities and hate speech. The second major way that social media influences individuals towards violence is through the unique ways in which these groups function on social media. For incels, these online communities provided a sense of community and belonging for those with mental health issues and loneliness. Incel communities also presented a propensity to idolise mass killers. The alt-right also shared this aspect, as 30 terrorists are idolised and future attacks are encouraged. The alt-right also had its own unique set of themes which included memes and trolling. These themes will be identified in the cases of Alek Minassian and Payton Gendron in the next section. Case analysis The main themes identified in the literature regarding incels are: mental health issues, loneliness, and the deification of mass killers. In the case of incels, those who take the “red pill” argue that they have transcended to a state of awareness about society, a truth that is depressing in nature. This truth includes the idea that women hold power over men, and many also go as far as arguing that incels are oppressed by both attractive men and all women, leaving just the unattractive incels on the bottom. Through this logic, acts of violent revenge are seen as justifiable. The themes that will emerge in the Alek Minassian case are mental health issues (specifically, his autism diagnosis), loneliness, and Minassian’s obsession with mass killers, especially Elliot Rodger. The case used for the alt-right is Payton Gendron, the perpetrator of the Tops supermarket shooting in Buffalo, New York. The manifesto he posted online before his shooting detailed his process of radicalisation through social media, and themes such as trolling, memes, and deification of terrorists are identified. Alek Minassian Alek Minassian, on April 23 of 2018, rented a van which he shortly after used to intentionally ram into pedestrians in downtown Toronto (Boyko, 2021). This tragedy caused the deaths of 10 people and injured another 16 additional people. This attack has been widely referred to as an incel attack, as Minassian has confirmed and detailed himself in a lengthy police interview after his arrest. This interview with the Toronto Police was one of the sources of data for this case analysis, as the interviewer inquires about Minassians life, mental health struggles, 31 his online activity, and ideology - all helpful in investigating the link between online activity and terrorism. The bigger source of data used was court documents from the Superior Court of Justice of Ontario. This document goes into even more detail about how Minassian’s Autism diagnosis has affected his life, which is a crucial aspect of this case as he attempted to use Autism Spectrum Disorder as a legal defence for his actions (R. v. Minassian, 2021). The document also covers Minassian’s life and struggles as well as some of his online activity. Earlier, in the literature review, it was revealed that Autism, both officially diagnosed and self-diagnosed, was significantly more prevalent among those who are engaged in incel activity online, at least twice as much (Moskalenko et al., 2022). Regehr (2020), as stated earlier, believes that incel spaces functioning as a place of belonging is a key feature. And as Sparks et al. (2022) report, ASD being prevalent in the population sheds light on experiences incels have with social isolation. Clearly, it is suggested by the literature that Autism is at least a factor, among other things, that contribute to an overall sense of social isolation - which may be why incels seek a place of social belonging in incel communities. So, going back to the goal of the current paper, understanding how autism affected Minassian’s life elucidates how online spaces provided a venue for him to become radicalised. Other themes to be discussed relating to Minassian include his reverence of previous mass killers, his struggle with Autism, his involvement in certain websites where incel and alt-right content is prevalent, and his immersion in online incel subcultures. Throughout the analysis, Regehr’s (2020) 5 step indoctrination process, which focuses on the conversion of loneliness into anger, will be apparent and referred to. According to R. v. Minassian (2021), Minassian has Autism Spectrum Disorder and one of the biggest challenges that he struggled with as a result was social skills and communication. 32 This started at an extremely young age, where he had difficulty communicating with other children in kindergarten (para. 89). In highschool, he faced bullying (para. 92), and his social interaction issues persisted, which manifested in the forms of being unable to interpret facial expression, tone of voice, and understanding other people’s emotions (para. 108). These struggles were described by a psychologist as life-long struggles which prevented him from having meaningful relationships with friends and romantic involvements (para. 108). One psychologist who testified in this case, Dr. Westphal, went as far as saying that Minassian’s communication skills were that of a 2-year-old child (para. 109). However, this claim was criticised by the other psychologists and even rejected by the judge (para. 115). Minassian was said to also be “profoundly lonely, “a significant theme in the case which was also used by the judge as a factor for why he believes Minassian committed this attack (para. 130, 196). In relation to Minassian’s online activity, several comments were made. For example Dr. Westphal stated: No doubt Mr. Minassian’s plan was conceived in the context of his saturation with provocative, hate-filled material on the internet. ASD made him less able to appreciate the theatrical, exaggerated nature and extremeness of the material, or the dark humor behind some of it. He took it very literally. (para. 134) The judge was critical of this claim, because, he argues, Minassian has said that people on this site were joking, and warned that the site does “joke” about mass killers and that it is mostly “satire” (para. 135). However, these two facts do not have to be mutually exclusive; as the literature review revealed, there is a subculture on sites like 4Chan that encourages what scholars call an “exploratory play frame” where things are treated as true and not true at the same time, as “serious” and “not serious” at the same time (Thorleifsson, 2022). DeCook (2020) also describes 33 trolling as a political aesthetic, where blatantly offensive things are treated as a joke but can also be serious at the same time, because the alt-right can claim they are joking when they are confronted with actual consequences. So, I believe the judge was too focused on the issue of whether Minassian took the “dark humour” literally and overlooked the fact that they may have still had a profound influence on this thought process either way. But it is understandable, as this case was primarily concerned with whether Minassian can use Autism as a legal defence for his actions. In the judge’s analysis of the case, specifically to come up with a reason for the attack, the judge argues that identification with incelism was not a “primary driving force” (para. 193). Instead, the judge argues that faced with a fear of failure in his job and social relationships as well as a desire to be seen and known drove him to commit the act for fame and notoriety (para. 196). However, this conclusion seems to be overly dismissive of the influence that incel subculture had on Minassian. The judge goes as far as to say that Minassian’s claim to the police that he wanted to start an incel rebellion is a “lie” (para. 195). The goal of the following discussion is not intended to “disprove” the judge’s opinion on why Minassian committed his attack, that is beyond the scope of this paper. The goal here is to establish a connection between Minsassian’s online activity and his actions, specifically how the former influenced the other. It is clear that Minassian’s immersion in incel culture was at least somewhat influential. Minassian himself gave an estimate that 50% of the reason he committed the attack was because of the incel ideology (para. 173). A psychiatrist, Dr. Woodside, placed “anger at women” at 35% (para. 172). Dr. Westphal claimed it was 40% identification with Elliot Rodger and only 20% notoriety and fame and 15% anxiety about his job. However, Dr. Westphal claims that the incel 34 claim was only a “veneer” (para. 186) Minassian also told Dr. Westphal that if he had the opportunity to do this attack again, he would try to target women between the ages of 18 and 30. However, Minassian did tell Dr. Westphal specifically, that he only mentioned the “incel rebellion” as a way to get more notoriety for himself. One interesting thing mentioned in this case is that Minassian mentioned to one of the assessors that he purposely tried to bait the police into shooting him after the attack in order to be “martyrized” and getting more attention (para. 175). This is reminiscent of Regehr’s (2021) final step in the indoctrination process, which is the deification and martyrdom of mass killers, where mass killers are documented and celebrated as well as encouraged online, creating a cyclical process. Another point in the Minassian case, which also elucidates Regehr’s (2020) “deification” of mass killers step in the indoctrination process, is his idolization of Elliot Rodger. Throughout the trial, it was mentioned that Minassian was obsessed with mass killings, and spent his time on sites dedicated to information about mass killers and their statistics, especially their “kill counts” (para. 132, 187). Minassian was obsessed with Elliot Rodger and his manifesto to the point of obsession (para. 132). He even said in his police interview that Rodger’s attack was an event specifically that “radicalised him” (para. 143). He posted on his FaceBook account, shortly before his attack (para. 149): Private (Recruit) Minassian Infantry 00010, wishing to speak to Sgt 4chan please. C23249161. The Incel Rebellion has already begun! We will overthrow all the Chads and Stacys! All hail the Supreme Gentleman Elliot Rodger! Another assessor of Minassian, Dr. Chouhan claimed that Minassian identified with Rodger in an unusual way, where he was seeing no differences between them, only similarities (para. 162). 35 Minassian was obsessed with Rodger to the point that he would think and read about him daily (para. 162). One final note about Minassian is his nuanced knowledge of the incel ideology. In the aforementioned FaceBook post he made shortly before his attack, he uses words like “Chads” and “Stacey’s,” mentions a rebellion as well as the site 4Chan. Regehr (2021) says these terms are used to refer to who incels seek to “deride and destroy,” as they are individuals who have more social and sexual power than the incels. Minassian also says in the police interview footage that he frequented specific sites like /b/ (random) /pol/ (politically incorrect) and /R9K/ (Robot 9000, Attorney Review, 1:17:30, 2020). At minute 1:33:50, Minassian describes the goals of the incel movement, stating that they want to overthrow the Chads, so that the Staceys are forced to reproduce with us [incels] (Attorney Review, 2020). To conclude, it is evident that Minassian was influenced by his interactions online through 3 major things. First, his idolization of mass killers and his wish to be “martyrized” himself. Second, his nuanced knowledge of the incel ideology, which is apparent from terminology that is subculture-specific. Finally, his involvement in websites like 4Chan, one of the ways in which he was familiarised with incel ideology and interacted with others. Payton Gendron Gendron is the person who was behind the egregious 2021 attack in Buffalo, New York in which he targeted black people as his victims, killing 10 people and wounding others in a supermarket. This section will use Gendron’s manifesto as the main source of data, as he goes into detail about his radicalisation process. The key themes revealed in the literature that will be discussed here is the glorification of terrorists, memes, and trolling. Gendron starts his manifesto off by saying: 36 If there’s one thing I want you to get from these writings, it’s that White birth rates must change. Everyday the White population becomes fewer in number (p. 1) Concern about white birth rates is directly related to the “white genocide” or “great replacement” alt-right conspiracy theory, which were the defining feature of this manifesto (Thorleifsson, 2021). Gendron’s manifesto starts with a Q&A-styled section, where he answers the questions that he believes will be asked by people. This provides a lot of insights to his ideology. It is of no doubt that he perpetrated this attack on the basis of alt-right ideology: He answers the question “why did you decide to carry out this attack?” with several sentences, many of them relating to inciting violence, intimidating “invaders” (people of colour), and spreading awareness to fellow white people about the “real” problems that the west faces (Gendron, p. 4, 2021). He claims he wants to “ensure the existence of our [white] people (Gendron, p. 4). He acknowledges it as an act of terrorism (p. 6). Refers to non-white people as invaders (p. 6, p. 54, 159, 161, 165, 166, 170). He is also explicitly and violently anti-semitic, when answering the question “are you antisemitic? He replies with “YES!! I wish all JEWS to HELL! Go back to hell where you came from DEMON!...” (p. 7). Gendron also goes into conspiracy theories about Jews and how they control the media, the government and global banking (p.24, 57). He believes that Jews are behind many of the problems faced in the west, and that they seek to turn white people on eachother (p. 53). The only core alt-right ideology that is essentially absent is misogyny. Gendron seems to be preoccupied with the idea that white people are being replaced, but does not mention his views on women or feminism. It is probably right to infer, however, that he believes in traditional gender roles, as he does mention the “traditional family unit” Memes appear quite frequently in Gendron’s manifesto. One of the memes associated with 4Chan and the alt-right is the “Happy Merchant” meme, which is a cartoon portrayal of a 37 heavily-stereotyped Jewish man (Thorleifsson, 2021). This meme, which is popular amongst the alt-right on 4Chan, appears quite frequently in Gendron’s manifesto. The first instance is on page 32, where Gendron uses a “shadow” version of the meme, where only the outline of the man is visible, and there are several words written on the man like: The Elite, The 1%, Globalists, The Media, Wall Street, Politicians, Internationalist among other things (p. 32). This meme is likely communicating the message that Jews “control” a lot of world affairs. The second instance appears on page 35, where the same meme appears, but this time it portrays what looks to be a stereotypical British man, with a monocle, tophat, and a cup of tea by his side. Behind the man appears a news article titled: “LONDON: Capital of World Jewry.” The third instance is on page 36, where the meme appears three times and all of them are in their original format (instead of a variation). The first two memes appear in an overall piece that Gendron shares which is titled “The ‘Smart Jew’ Myth: A lie propagated by Jews for the benefit of Jews.” The final meme appears kind of randomly, as it does not communicate anything directly, but it is placed within the context of other posts that Gendron included that have antisemitic dialogue (p. 36). The next use of this meme appears on page 45, where Gendron uses the standard format of the meme along with this text: He he yes white goyim give your children to me and I’ll make them think they’re girls. He he thus they’ll all either kill themselves or become genetic dead ends. Oy vey goyim… it’s just too easy. This meme here is implying that Jewish people are somehow propagating transgenderism in a ploy to reduce the white birth rates, as it mentions “genetic dead ends” (p. 45). On page 47, the meme appears again, this time in a colourised version, and the Jewish man is seen holding a heavily-stereotyped drawing of a black man and placing him into a funnel. This funnel is placed 38 inside of a white man’s mouth (who is depicted being tied by rope and held down by a pair of hands). The Jewish man has “Television, Magazines, Radio, Media, Music” written on him. Again, this meme is communicating the message that Jewish people control the media and are using it as a tool to “make” white people accept people of colour, which presumably Gendron believes will lead to the replacement of white people. On page 48, Gendron shares the happy merchant meme in the format of a Jewish woman this time, and behind her sits a news article that denounces white supremacy. On page 50, a heavily stereotyped cartoon of a Jewish man appears, although it is not exactly the “Happy Merchant” meme. This man is depicted pouring a liquid, labelled with things like “guilt, confusion, pornography, paranoia” into the well of “white culture. The final appearance of the “Happy Merchant” meme appears on page 53, where he uses the depiction of the heavily-stereotyped Jewish man to claim that the parent pornography company “MindGeek” owns several porn websites like PornHub, RedTube, Youporn, and Brazzers. This portrayal is not unlike the previous ones, where pornography is seen as a weapon used by the Jews to orchestrate an attack on the white race. An instance of trolling appears on the third page, where Gendron posts pictures of a man and claims they are his most recent pictures. However, the pictures he includes are not him, they are that of an individual known as “Sam Hyde.” Sam Hyde is not a mass shooter, but an internet comedian, and people online accuse him of every mass shooting after they occur as part of a long-running hoax to trick the media (Bromwich, 2017). This is not unlike the “ok” hand signal hoax, where 4Chan attempted to troll the media by claiming the hand gesture was in fact a white supremacist hand signal, believing the media is gullible enough to believe this (DeCook, 2020). On analysis of Gendron’s activity online and how it influenced him, two themes emerge. First, his immersion in 4Chan alt-right subculture (memes) and second, his idolization of 39 previous terrorists. Conveniently, Gendron has gone into detail about his radicalization in the Q&A section (p. 13), where the question is “Was there a particular event or reason you decided to commit a violent attack?” to which he responds: Before I begin I will say that I was not born racist nor grew up to be racist. I simply became racist after I learned the truth. I started browsing 4chan in May 2020 after extreme boredom, remember this was during the outbreak of covid. I would normally browse /k/ because I’m a gun nut and /out/ because I love the outdoors and I eventually wound up on /pol/. There I learned through infographics, shitposts, and memes that the White race is dying out, that blacks are disproportionately killing Whites, that the average black takes $700,000 from tax-payers in their lifetime, and that the Jews and the elite were behind this. From there, I also found other sites, like worldtruthvideos.website, dailyarchives.org, and dailystormer.cn where through data and exposure to real information I learned the truth. We are doomed by low birth rates and high rates of immigration. I never even saw this information until I found these sites, since mostly I would get my news from the front page of Reddit. I didn’t care at the time, but as I learned more and more I realized how serious the situation was. Eventually I couldn’t take it anymore, I told myself that eventually I was going to kill myself to escape this fate. My race was doomed and there was nothing I could do about it. But then after browsing /pol/ one day I saw a short gif of a man walking into a building and shooting a shotgun through a dark hallway. I didn’t think anything of it, but then I saw it again, and I looked up who this person was. I thought to myself, “Why did this person do it?” That person was Brenton Tarrant, and after some searches I found the 17 minute livestream of him attacking the Al-Noor mosque. I eventually found his manifesto and I 40 read it, and I found that I mostly agreed with him. Finally I thought to myself, perhaps there is a chance that we can combat this. Maybe there is a chance that we can take control and prevent our genocide. Maybe we can combat the hedonistic, nihilistic, and individualist insanity that has taken control western thought. I then found other fighters, like Patrick Crucius, Anders Breivek, Dylann Roof, and John Earnest. These men fought for me and had the same goals I did. It was there I asked myself: Why don’t I do something? (p.13) This lengthy response provides a lot of insight into his radicalization process and the influence that social media sites like 4Chan had on him. He mentions how he learnt the alt-right ideology through memes and shitposts on 4Chan. The other interesting aspect is his idolization of Tarrant, the Christchurch mosque shooter along with other terrorists like Breivik, Earnest, and Crucius. Revering previous terrorists to the point of sainthood is a theme common among the alt-right on 4Chan (Thorleifsson, 2021). He states this explicitly in another Q&A question, “Is there a particular person that radicalised you the most.” He responds: Yes and his name is Brenton Harrison Tarrant. Brenton’s livestream started everything you see here. Brenton started my real research into the problems with immigration and foreigners in our White lands, without his livestream I would likely have no idea about the real problems the West is facing. (p. 8) There are a few takeaways from the Gendron case. First, the self-proclaimed reason he became radicalised was due to browsing the 4chan online image board. He learned about alt-right conspiracies through memes and shitposts he found on this site. He idolised Christchurch shooter Tarrant and other such terrorists after seeing a video of Tarrant’s live streamed attack on the 41 mosque. His online activity, within a span of a year, turned him from a “normal person” to someone who was ready to commit acts of violence. The first major theme that emerges in Gendron’s case is his immersion in 4Chan subculture. His use of alt-right memes like the “Happy Merchant” shows a level of deep immersion and understanding of 4Chan subculture. These very memes, which he uses in his own manifesto to spread this hateful ideology, come from 4Chan. These are the types of memes that were identified in the literature review, and he admits that they radicalised him into the alt-right ideology. Another indicator of his immersion in 4Chan subculture is the theme of trolling. It happens at least once, where he uses the Sam Hyde hoax. This type of behaviour was identified in the literature review and is a core aspect of the alt-right on 4Chan. The second major influence that social media had on Gendron is the idolisation of terrorists. He explicitly confesses his reverence to the Christchurch shooter in his manifesto, and cites seeing a video of this shooting as a major influence on his radicalisation. This suggests two things. First, memes on social media are very effective and influential in spreading hateful ideologies. Second, social media makes it possible for mass shooters to be celebrated and idolised in a unique way, such as by spreading videos and memes of shootings, ultimately glorifying them. Conclusion The findings from the literature review show that the “attention economy” led to certain design features being included in many social media sites. These very design features can promote the emergence of toxic practices and communities online. Common themes emerged from the discussion of incels and the alt-right communities online, such as the idolization of terrorists, memes, shitposting, mental health and trolling. One theme that emerged with incels specifically is the function of online spaces as providing a sense of belonging to those who 42 struggle with loneliness and mental health issues, specifically autism. Although both incel terrorists and alt-right terrorists commit their acts for different reasons, their involvement and immersion in online subcultures are a driving force behind their radicalisation. It is not that these individuals with hateful ideologies seek these spaces to discuss their hate with others and then become more radicalised to the point of seeking to commit violence. Rather, these spaces function as a community where these individuals come in with relatively mild views, if at all, and are systematically indoctrinated through “joking” memes, trolling, and shitposting. Ultimately, an individual’s participation in the community, which may have started for a reason as simple as being “bored” (as in Gendron’s case), ends up inducing a sense of urgency for the need to commit violence. Or in the case of incels, these communities transform, as Regehr (2020) argues, loneliness into anger, making men feel as if they are being oppressed by society. And by idolising those who “stand up” and attempt to overthrow the “unjust system,” this behaviour is encouraged and celebrated, creating a cycle (Regehr, 2020). These themes were prevalent in the case analyses. In the case of Alek Minassian, his struggle with autism spectrum disorder was lengthy and it contributed to him being very socially isolated, as evidenced by court reports. He also displayed a level of understanding of incel ideology in his police interview, citing the “incel rebellion” as his motivation. This indicates that he was active within incel communities in some form. As the literature review suggests, incel ideologies convert men’s loneliness into misogynistic anger. Minassian also displayed a high level of idolisation and obsession with Elliot Rodger, the incel mass killer commonly identified as the first incel mass killer. In the case of Gendron, there are two major themes identified. First, his immersion in 4Chan subculture as evidenced by his understanding of alt-right memes and trolls such as the Sam Hyde hoax. He states this explicitly himself in his manifesto, citing memes as the principal 43 way he learned about alt-right conspiracy theories. The second major theme identified is the idolisation of terrorists. He also explicitly states this in his manifesto, citing the Christchurch shooter as his inspiration. The findings suggest that social media, through design, policies, and the way they function (through memes, trolling, and celebrating mass killers) can encourage taking violent action in real life. 44 References Attorney Review. (2020, February, 28). Full police interrogation of deadly van attacker driver Alek Minassian [video]. https://www.youtube.com/watch?v=nbJzcsLzSNo Anderson, L.& Barnes, M. (2022). Hate speech. The Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/archives/spr2022/entries/hate-speech Baele, S. J., Brace, L. and Coan T. G. (2019) From “incel” to “saint”: Analyzing the violent worldview behind the 2018 Toronto attack. Terrorism and Political Violence, 33(8), 1667-1691. https://doi.org/10.1080/09546553.2019.1638256 Baele, S. J., Brace, L., & Ging, D. (2023). A diachronic cross-platforms analysis of violent extremist language in the incel online ecosystem. Terrorism and Political Violence, 1–24. https://doi.org/10.1080/09546553.2022.2161373 Bhargava, V. R. & Velasquez, M. (2021). Ethics of the attention economy: The problem of social media addiction. Business Ethics Quarterly, 31(3), 321–359. https://doi.org/10.1017/beq.2020.32 Boucher, V. (2022). Down the TikTok rabbit hole: Testing the TikTok algorithm’s contribution to right wing extremist radicalization. Queen’s University. https://www.proquest.com/dissertations-theses/down-tiktok-rabbit-hole-testing-algorithm -s/docview/2723856317/se-2 Boyko, J. (2021, August 13). 2018 Toronto Van Attack. The Canadian Encyclopedia. https://www.thecanadianencyclopedia.ca/en/article/2018-toronto-van-attack Bromwich, J. E. (2017, November 6). Sam Hyde and Other Hoaxes: False Information Trails Texas Shooting. https://www.nytimes.com/2017/11/06/us/shooting-texas-hoaxes.html 45 Davis, M. (2023). After Christchurch: Alt-right discourse and ideology in Australia and the platformisation of extremism. Journal of Intercultural Studies, 18(1), 1–17. https://doi.org/10.1080/07256868.2023.2149479 DeCook, J. R. (2020). Trust me, I’m trolling: Irony and the alt-right’s political aesthetic. M/C Journal, 23(3). https://doi.org/10.5204/mcj.1655 Gaudette, T., Scrivens, R., Davies, G., & Frank, R. (2021). Upvoting extremism: Collective identity formation and the extreme right on reddit. New Media & Society, 23(12), 3491–3508. https://doi.org/10.1177/1461444820958123 Gendron, P. (2021). Payton Gendron Manifesto. https://hushbimbo.com/2022/05/16/payton-gendron-manifesto/ Golf-Papez, & Veer, E. (2017). Don’t feed the trolling: rethinking how online trolling is being defined and combated. Journal of Marketing Management, 33(15-16), 1336–1354. https://doi.org/10.1080/0267257X.2017.1383298 Hodge, E. & Hallgrimsdottir, H. (2020). Networks of Hate: The alt-right, “troll culture”, and the cultural geography of social movement spaces. Online Journal of Borderlands Studies, 35(4), 563–580. https://doi.org/10.1080/08865655.2019.1571935 Institute for Strategic Dialogue. (2022). Accelerationism. https://www.isdglobal.org/explainers/accelerationism/ Kilvington. (2021). The virtual stages of hate: Using Goffman’s work to conceptualise the motivations for online hate. Media, Culture & Society, 43(2), 256–272. https://doi.org/10.1177/0163443720972318 Lindsay, A. (2020). Swallowing the black pill: A qualitative exploration of incel antifeminism within digital society. Victoria University of Wellington. 46 https://researcharchive.vuw.ac.nz/xmlui/bitstream/handle/10063/8915/thesis_access.pdf?s equence=1 Lindsay, A. (2022). Swallowing the black pill: Involuntary celibates’ (Incels) anti-feminism within digital society. International Journal for Crime, Justice and Social Democracy, 11(1), 210–224. https://doi.org/10.5204/ijcjsd.2138 Massanari, A. (2016). Gamergate and the fappening: How Reddit's algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3), 329–346. https://doi.org/10.1177/1461444815608807 Mezzofiore, G. & O’Sullivan, D. (2019, August 5). El Paso mass shooting is at least the third atrocity linked to 8chan this year. CNN. https://www.cnn.com/2019/08/04/business/el-paso-shooting-8chan-biz/index.html Montag, C., Lachmann, B., Herrlich, M., & Zweig, K. (2019). Addictive features of social media/messenger platforms and freemium games against the background of psychological and economic theories. International Journal of Environmental Research and Public Health, 16(14), 2612. https://doi.org/10.3390/ijerph16142612 Moskalenko, S. González, J. F.-G., Kates, N., & Morton, J. (2022). Incel ideology, radicalization and mental health: A survey study. The Journal of Intelligence, Conflict, and Warfare, 4(3), 1–29. https://doi.org/10.21810/jicw.v4i3.3817 O’Malley, R. L., Holt, K., & Holt, T. J. (2022). An exploration of the involuntary celibate (Incel) subculture online. Journal of Interpersonal Violence, 37(7-8), NP4981–NP5008. https://doi.org/10.1177/0886260520959625 Perryer, S. (2020, January 15). A sorry site: 8chan gets the axe, raising questions about internet censorship. The New Economy. 47 https://www.theneweconomy.com/technology/a-sorry-site-8chan-gets-the-axe-raising-que stions-about-internet-censorship Regehr, K. (2022). In(cel)doctrination: How technologically facilitated misogyny moves violence off screens and on to streets. New Media & Society, 24(1), 138–155. https://doi-org.libproxy.mtroyal.ca/10.1177/1461444820959019 R. v. Minassian, 2021 ONSC 1258 Shifman, L. (2014). Memes in Digital Culture. Cambridge, MA: MIT Press. Sparks, Zidenberg, A. M., & Olver, M. E. (2023). One is the loneliest number: Involuntary celibacy (incel), mental health, and loneliness. Current Psychology, 1–15. https://doi.org/10.1007/s12144-023-04275-z Sparks, B., Zidenberg, A. M. & Olver, M. E. (2022). Involuntary celibacy: A review of incel ideology and experiences with dating, rejection, and associated mental health and emotional sequelae. Current Psychiatry Reports, 24, 731–740 (2022). https://doi.org/10.1007/s11920-022-01382-9 Stijelja, S. & Mishara, B. L. (2023). Psychosocial characteristics of involuntary celibates (incels): A review of empirical research and assessment of the potential implications of research on adult virginity and late sexual onset. Sexuality & Culture, 27(2), 715–734. https://doi.org/10.1007/s12119-022-10031-5 Thompson, C., Wawrow, J., Balsamo, M. & Collins, D. (2022, May 14). AP News. https://apnews.com/article/buffalo-supermarket-shooting-442c6d97a073f39f99d006dbba 40f64b 48 Thorleifsson, C. (2022). From cyberfascism to terrorism: On 4chan/pol/ culture and the transnational production of memetic violence. Nations and Nationalism, 28(1), 286–301. https://doi.org/10.1111/nana.12780 Williams, D. J., Arntfield, M., Schaal, K., & Vincent, J. (2021). Wanting sex and willing to kill: Examining demographic and cognitive characteristics of violent “involuntary celibates”. Behavioral Sciences & the Law, 39(4), 386– 401. https://doi.org/10.1002/bsl.2512 Wendling, W. (2019, August 5). What is 8Chan? BBC News. https://www.bbc.com/news/blogs-trending-49233767 Zhang, & Luo, L. (2018). Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter. https://doi.org/10.48550/arxiv.1803.03662 4Chan. (2023). Frequently asked questions. Retrieved April 4, 2023 from https://www.4channel.org/faq