Hands-on-Exercise-05

27  Chapter 27: Modelling, Visualising and Analysing Network Data with R

27.1 Overview

In this hands-on exercise, you will learn how to model, analyse and visualise network data using R.

By the end of this hands-on exercise, you will be able to:

• create graph object data frames, manipulate them using appropriate functions of dplyr, lubridate, and tidygraph,
• build network graph visualisation using appropriate functions of ggraph,
• compute network geometrics using tidygraph,
• build advanced graph visualisation by incorporating the network geometrics, and
• build interactive network visualisation using visNetwork package.

27.2 Getting Started

27.2.1 Installing and launching R packages

In this hands-on exercise, four network data modelling and visualisation packages will be installed and launched. They are igraph, tidygraph, ggraph and visNetwork. Beside these four packages, tidyverse and lubridate, an R package specially designed to handle and wrangling time data will be installed and launched too.

The code chunk:

pacman::p_load(igraph, tidygraph, ggraph, 
               visNetwork, lubridate, clock,
               tidyverse, graphlayouts, 
               concaveman, ggforce)

27.3 The Data

The data sets used in this hands-on exercise is from an oil exploration and extraction company. There are two data sets. One contains the nodes data and the other contains the edges (also know as link) data.

27.3.1 The edges data

• GAStech-email_edges.csv which consists of two weeks of 9063 emails correspondances between 54 employees.

27.3.2 The nodes data

• GAStech_email_nodes.csv which consist of the names, department and title of the 54 employees.

27.3.3 Importing network data from files

In this step, you will import GAStech_email_node.csv and GAStech_email_edges-v2.csv into RStudio environment by using read_csv() of readr package.

GAStech_nodes <- read_csv("data/GAStech_email_node.csv")
GAStech_edges <- read_csv("data/GAStech_email_edge-v2.csv")

27.3.4 Reviewing the imported data

Next, we will examine the structure of the data frame using glimpse() of dplyr.

glimpse(GAStech_edges)

Rows: 9,063
Columns: 8
$ source      <dbl> 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 26, 26, 26…
$ target      <dbl> 41, 40, 51, 52, 53, 45, 44, 46, 48, 49, 47, 54, 27, 28, 29…
$ SentDate    <chr> "6/1/2014", "6/1/2014", "6/1/2014", "6/1/2014", "6/1/2014"…
$ SentTime    <time> 08:39:00, 08:39:00, 08:58:00, 08:58:00, 08:58:00, 08:58:0…
$ Subject     <chr> "GT-SeismicProcessorPro Bug Report", "GT-SeismicProcessorP…
$ MainSubject <chr> "Work related", "Work related", "Work related", "Work rela…
$ sourceLabel <chr> "Sven.Flecha", "Sven.Flecha", "Kanon.Herrero", "Kanon.Herr…
$ targetLabel <chr> "Isak.Baza", "Lucas.Alcazar", "Felix.Resumir", "Hideki.Coc…

27.3.5 Wrangling time

The code chunk below will be used to perform the changes.

GAStech_edges <- GAStech_edges %>%
  mutate(SendDate = dmy(SentDate)) %>%
  mutate(Weekday = wday(SentDate,
                        label = TRUE,
                        abbr = FALSE))

Things to learn from code chunk above

• both dmy() and wday() are functions of lubridate package. lubridate is an R package that makes it easier to work with dates and times.
• dmy() transforms the SentDate to Date data type.
• wday() returns the day of the week as a decimal number or an ordered factor if label is TRUE. The argument abbr is FALSE keep the daya spells in full, i.e. Monday. The function will create a new column in the data.frame i.e. Weekday and the output of wday() will save in this newly created field.
• the values in the Weekday field are in ordinal scale.

27.3.6 Reviewing the revised date fields

Table below shows the data structure of the reformatted GAStech_edges data frame

glimpse(GAStech_edges)

Rows: 9,063
Columns: 10
$ source      <dbl> 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 26, 26, 26…
$ target      <dbl> 41, 40, 51, 52, 53, 45, 44, 46, 48, 49, 47, 54, 27, 28, 29…
$ SentDate    <chr> "6/1/2014", "6/1/2014", "6/1/2014", "6/1/2014", "6/1/2014"…
$ SentTime    <time> 08:39:00, 08:39:00, 08:58:00, 08:58:00, 08:58:00, 08:58:0…
$ Subject     <chr> "GT-SeismicProcessorPro Bug Report", "GT-SeismicProcessorP…
$ MainSubject <chr> "Work related", "Work related", "Work related", "Work rela…
$ sourceLabel <chr> "Sven.Flecha", "Sven.Flecha", "Kanon.Herrero", "Kanon.Herr…
$ targetLabel <chr> "Isak.Baza", "Lucas.Alcazar", "Felix.Resumir", "Hideki.Coc…
$ SendDate    <date> 2014-01-06, 2014-01-06, 2014-01-06, 2014-01-06, 2014-01-0…
$ Weekday     <ord> Friday, Friday, Friday, Friday, Friday, Friday, Friday, Fr…

27.3.7 Wrangling attributes

A close examination of GAStech_edges data.frame reveals that it consists of individual e-mail flow records. This is not very useful for visualisation.

In view of this, we will aggregate the individual by date, senders, receivers, main subject and day of the week.

The code chunk:

GAStech_edges_aggregated <- GAStech_edges %>%
  filter(MainSubject == "Work related") %>%
  group_by(source, target, Weekday) %>%
    summarise(Weight = n()) %>%
  filter(source!=target) %>%
  filter(Weight > 1) %>%
  ungroup()

Things to learn from code chunk above

• four functions from dplyr package are used. They are: filter(), group(), summarise(), and ungroup().
• The output data.frame is called GAStech_edges_aggregated.
• A new field called Weight has been added in GAStech_edges_aggregated.

27.3.8 Reviewing the revised edges file

Table below shows the data structure of the reformatted GAStech_edges data frame

glimpse(GAStech_edges_aggregated)

Rows: 1,372
Columns: 4
$ source  <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
$ target  <dbl> 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 6,…
$ Weekday <ord> Sunday, Monday, Tuesday, Wednesday, Friday, Sunday, Monday, Tu…
$ Weight  <int> 5, 2, 3, 4, 6, 5, 2, 3, 4, 6, 5, 2, 3, 4, 6, 5, 2, 3, 4, 6, 5,…

27.4 Creating network objects using tidygraph

In this section, you will learn how to create a graph data model by using tidygraph package. It provides a tidy API for graph/network manipulation. While network data itself is not tidy, it can be envisioned as two tidy tables, one for node data and one for edge data. tidygraph provides a way to switch between the two tables and provides dplyr verbs for manipulating them. Furthermore it provides access to a lot of graph algorithms with return values that facilitate their use in a tidy workflow.

Before getting started, you are advised to read these two articles:

• Introducing tidygraph

• tidygraph 1.1 - A tidy hope

27.4.1 The tbl_graph object

Two functions of tidygraph package can be used to create network objects, they are:

• tbl_graph() creates a tbl_graph network object from nodes and edges data.

• as_tbl_graph() converts network data and objects to a tbl_graph network. Below are network data and objects supported by as_tbl_graph()

  • a node data.frame and an edge data.frame,

  • data.frame, list, matrix from base,

  • igraph from igraph,

  • network from network,

  • dendrogram and hclust from stats,

  • Node from data.tree,

  • phylo and evonet from ape, and

  • graphNEL, graphAM, graphBAM from graph (in Bioconductor).

27.4.2 The dplyr verbs in tidygraph

• activate() verb from tidygraph serves as a switch between tibbles for nodes and edges. All dplyr verbs applied to tbl_graph object are applied to the active tibble.

• In the above the .N() function is used to gain access to the node data while manipulating the edge data. Similarly .E() will give you the edge data and .G() will give you the tbl_graph object itself.

27.4.3 Using tbl_graph() to build tidygraph data model.

In this section, you will use tbl_graph() of tinygraph package to build an tidygraph’s network graph data.frame.

Before typing the codes, you are recommended to review to reference guide of tbl_graph()

GAStech_graph <- tbl_graph(nodes = GAStech_nodes,
                           edges = GAStech_edges_aggregated, 
                           directed = TRUE)

27.4.4 Reviewing the output tidygraph’s graph object

GAStech_graph

# A tbl_graph: 54 nodes and 1372 edges
#
# A directed multigraph with 1 component
#
# Node Data: 54 × 4 (active)
      id label               Department     Title                               
   <dbl> <chr>               <chr>          <chr>                               
 1     1 Mat.Bramar          Administration Assistant to CEO                    
 2     2 Anda.Ribera         Administration Assistant to CFO                    
 3     3 Rachel.Pantanal     Administration Assistant to CIO                    
 4     4 Linda.Lagos         Administration Assistant to COO                    
 5     5 Ruscella.Mies.Haber Administration Assistant to Engineering Group Mana…
 6     6 Carla.Forluniau     Administration Assistant to IT Group Manager       
 7     7 Cornelia.Lais       Administration Assistant to Security Group Manager 
 8    44 Kanon.Herrero       Security       Badging Office                      
 9    45 Varja.Lagos         Security       Badging Office                      
10    46 Stenig.Fusil        Security       Building Control                    
# ℹ 44 more rows
#
# Edge Data: 1,372 × 4
   from    to Weekday Weight
  <int> <int> <ord>    <int>
1     1     2 Sunday       5
2     1     2 Monday       2
3     1     2 Tuesday      3
# ℹ 1,369 more rows

27.4.5 Reviewing the output tidygraph’s graph object

• The output above reveals that GAStech_graph is a tbl_graph object with 54 nodes and 4541 edges.

• The command also prints the first six rows of “Node Data” and the first three of “Edge Data”.

• It states that the Node Data is active. The notion of an active tibble within a tbl_graph object makes it possible to manipulate the data in one tibble at a time.

27.4.6 Changing the active object

The nodes tibble data frame is activated by default, but you can change which tibble data frame is active with the activate() function. Thus, if we wanted to rearrange the rows in the edges tibble to list those with the highest “weight” first, we could use activate() and then arrange().

For example,

GAStech_graph %>%
  activate(edges) %>%
  arrange(desc(Weight))

# A tbl_graph: 54 nodes and 1372 edges
#
# A directed multigraph with 1 component
#
# Edge Data: 1,372 × 4 (active)
    from    to Weekday   Weight
   <int> <int> <ord>      <int>
 1    40    41 Saturday      13
 2    41    43 Monday        11
 3    35    31 Tuesday       10
 4    40    41 Monday        10
 5    40    43 Monday        10
 6    36    32 Sunday         9
 7    40    43 Saturday       9
 8    41    40 Monday         9
 9    19    15 Wednesday      8
10    35    38 Tuesday        8
# ℹ 1,362 more rows
#
# Node Data: 54 × 4
     id label           Department     Title           
  <dbl> <chr>           <chr>          <chr>           
1     1 Mat.Bramar      Administration Assistant to CEO
2     2 Anda.Ribera     Administration Assistant to CFO
3     3 Rachel.Pantanal Administration Assistant to CIO
# ℹ 51 more rows

Visit the reference guide of activate() to find out more about the function.

27.5 Plotting Static Network Graphs with ggraph package

ggraph is an extension of ggplot2, making it easier to carry over basic ggplot skills to the design of network graphs.

As in all network graph, there are three main aspects to a ggraph’s network graph, they are:

• nodes,

• edges and

• layouts.

For a comprehensive discussion of each of this aspect of graph, please refer to their respective vignettes provided.

27.5.1 Plotting a basic network graph

The code chunk below uses ggraph(), geom-edge_link() and geom_node_point() to plot a network graph by using GAStech_graph. Before your get started, it is advisable to read their respective reference guide at least once.

ggraph(GAStech_graph) +
  geom_edge_link() +
  geom_node_point()

Tip

• The basic plotting function is ggraph(), which takes the data to be used for the graph and the type of layout desired. Both of the arguments for ggraph() are built around igraph. Therefore, ggraph() can use either an igraph object or a tbl_graph object.

27.5.2 Changing the default network graph theme

In this section, you will use theme_graph() to remove the x and y axes. Before your get started, it is advisable to read it’s reference guide at least once.

g <- ggraph(GAStech_graph) + 
  geom_edge_link(aes()) +
  geom_node_point(aes())

g + theme_graph()

Note

• ggraph introduces a special ggplot theme that provides better defaults for network graphs than the normal ggplot defaults. theme_graph(), besides removing axes, grids, and border, changes the font to Arial Narrow (this can be overridden).

• The ggraph theme can be set for a series of plots with the set_graph_style() command run before the graphs are plotted or by using theme_graph() in the individual plots.

27.5.3 Changing the coloring of the plot

Furthermore, theme_graph() makes it easy to change the coloring of the plot.

g <- ggraph(GAStech_graph) + 
  geom_edge_link(aes(colour = 'grey50')) +
  geom_node_point(aes(colour = 'grey40'))

g + theme_graph(background = 'grey10',
                text_colour = 'white')

27.5.4 Working with ggraph’s layouts

ggraph support many layout for standard used, they are: star, circle, nicely (default), dh, gem, graphopt, grid, mds, spahere, randomly, fr, kk, drl and lgl. Figures below and on the right show layouts supported by ggraph().

27.5.5 Fruchterman and Reingold layout

The code chunks below will be used to plot the network graph using Fruchterman and Reingold layout.

g <- ggraph(GAStech_graph, 
            layout = "fr") +
  geom_edge_link(aes()) +
  geom_node_point(aes())

g + theme_graph()

Thing to learn from the code chunk above:

• layout argument is used to define the layout to be used.

27.5.6 Modifying network nodes

In this section, you will colour each node by referring to their respective departments.

g <- ggraph(GAStech_graph, 
            layout = "nicely") + 
  geom_edge_link(aes()) +
  geom_node_point(aes(colour = Department, 
                      size = 3))

g + theme_graph()

Tip

Things to learn from the code chunks above:

• geom_node_point is equivalent in functionality to geo_point of ggplot2. It allows for simple plotting of nodes in different shapes, colours and sizes. In the codes chnuks above colour and size are used.

27.5.7 Modifying edges

In the code chunk below, the thickness of the edges will be mapped with the Weight variable.

g <- ggraph(GAStech_graph, 
            layout = "nicely") +
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department), 
                  size = 3)

g + theme_graph()

Tip

Things to learn from the code chunks above:

• geom_edge_link draws edges in the simplest way - as straight lines between the start and end nodes. But, it can do more that that. In the example above, argument width is used to map the width of the line in proportional to the Weight attribute and argument alpha is used to introduce opacity on the line.

27.6 Creating facet graphs

Another very useful feature of ggraph is faceting. In visualising network data, this technique can be used to reduce edge over-plotting in a very meaning way by spreading nodes and edges out based on their attributes. In this section, you will learn how to use faceting technique to visualise network data.

There are three functions in ggraph to implement faceting, they are:

• facet_nodes() whereby edges are only draw in a panel if both terminal nodes are present here,

• facet_edges() whereby nodes are always drawn in al panels even if the node data contains an attribute named the same as the one used for the edge facetting, and

• facet_graph() faceting on two variables simultaneously.

27.6.1 Working with facet_edges()

In the code chunk below, facet_edges() is used. Before getting started, it is advisable for you to read it’s reference guide at least once.

set_graph_style()

g <- ggraph(GAStech_graph, 
            layout = "nicely") + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department), 
                  size = 2)

g + facet_edges(~Weekday)

27.6.2 Working with facet_edges()

The code chunk below uses theme() to change the position of the legend.

set_graph_style()

g <- ggraph(GAStech_graph, 
            layout = "nicely") + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department), 
                  size = 2) +
  theme(legend.position = 'bottom')
  
g + facet_edges(~Weekday)

27.6.3 A framed facet graph

The code chunk below adds frame to each graph.

set_graph_style() 

g <- ggraph(GAStech_graph, 
            layout = "nicely") + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department), 
                  size = 2)
  
g + facet_edges(~Weekday) +
  th_foreground(foreground = "grey80",  
                border = TRUE) +
  theme(legend.position = 'bottom')

27.6.4 Working with facet_nodes()

In the code chunkc below, facet_nodes() is used. Before getting started, it is advisable for you to read it’s reference guide at least once.

set_graph_style()

g <- ggraph(GAStech_graph, 
            layout = "nicely") + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department), 
                  size = 2)
  
g + facet_nodes(~Department)+
  th_foreground(foreground = "grey80",  
                border = TRUE) +
  theme(legend.position = 'bottom')

27.7 Network Metrics Analysis

27.7.1 Computing centrality indices

Centrality measures are a collection of statistical indices use to describe the relative important of the actors are to a network. There are four well-known centrality measures, namely: degree, betweenness, closeness and eigenvector. It is beyond the scope of this hands-on exercise to cover the principles and mathematics of these measure here. Students are encouraged to refer to Chapter 7: Actor Prominence of A User’s Guide to Network Analysis in R to gain better understanding of theses network measures.

g <- GAStech_graph %>%
  mutate(betweenness_centrality = centrality_betweenness()) %>%
  ggraph(layout = "fr") + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department,
            size=betweenness_centrality))
g + theme_graph()

Tip

Things to learn from the code chunk above:

• mutate() of dplyr is used to perform the computation.

• the algorithm used, on the other hand, is the centrality_betweenness() of tidygraph.

27.7.2 Visualising network metrics

It is important to note that from ggraph v2.0 onward tidygraph algorithms such as centrality measures can be accessed directly in ggraph calls. This means that it is no longer necessary to precompute and store derived node and edge centrality measures on the graph in order to use them in a plot.

g <- GAStech_graph %>%
  ggraph(layout = "fr") + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(colour = Department, 
                      size = centrality_betweenness()))
g + theme_graph()

27.7.3 Visualising Community

tidygraph package inherits many of the community detection algorithms imbedded into igraph and makes them available to us, including Edge-betweenness (group_edge_betweenness), Leading eigenvector (group_leading_eigen), Fast-greedy (group_fast_greedy), Louvain (group_louvain), Walktrap (group_walktrap), Label propagation (group_label_prop), InfoMAP (group_infomap), Spinglass (group_spinglass), and Optimal (group_optimal). Some community algorithms are designed to take into account direction or weight, while others ignore it. Use this link to find out more about community detection functions provided by tidygraph,

In the code chunk below group_edge_betweenness() is used.

g <- GAStech_graph %>%
  mutate(community = as.factor(
    group_edge_betweenness(
      weights = Weight, 
      directed = TRUE))) %>%
  ggraph(layout = "fr") + 
  geom_edge_link(
    aes(
      width=Weight), 
    alpha=0.2) +
  scale_edge_width(
    range = c(0.1, 5)) +
  geom_node_point(
    aes(colour = community))  

g + theme_graph()

In order to support effective visual investigation, the community network above has been revised by using geom_mark_hull() of ggforce package.

g <- GAStech_graph %>%
  activate(nodes) %>%
  mutate(community = as.factor(
    group_optimal(weights = Weight)),
         betweenness_measure = centrality_betweenness()) %>%
  ggraph(layout = "fr") +
  geom_mark_hull(
    aes(x, y, 
        group = community, 
        fill = community),  
    alpha = 0.2,  
    expand = unit(0.3, "cm"),  # Expand
    radius = unit(0.3, "cm")  # Smoothness
  ) + 
  geom_edge_link(aes(width=Weight), 
                 alpha=0.2) +
  scale_edge_width(range = c(0.1, 5)) +
  geom_node_point(aes(fill = Department,
                      size = betweenness_measure),
                      color = "black",
                      shape = 21)
  
g + theme_graph()

27.8 Building Interactive Network Graph with visNetwork

• visNetwork() is a R package for network visualization, using vis.js javascript library.

• visNetwork() function uses a nodes list and edges list to create an interactive graph.

  • The nodes list must include an “id” column, and the edge list must have “from” and “to” columns.

  • The function also plots the labels for the nodes, using the names of the actors from the “label” column in the node list.

• The resulting graph is fun to play around with.

  • You can move the nodes and the graph will use an algorithm to keep the nodes properly spaced.

  • You can also zoom in and out on the plot and move it around to re-center it.

27.8.1 Data preparation

Before we can plot the interactive network graph, we need to prepare the data model by using the code chunk below.

GAStech_edges_aggregated <- GAStech_edges %>%
  left_join(GAStech_nodes, by = c("sourceLabel" = "label")) %>%
  rename(from = id) %>%
  left_join(GAStech_nodes, by = c("targetLabel" = "label")) %>%
  rename(to = id) %>%
  filter(MainSubject == "Work related") %>%
  group_by(from, to) %>%
    summarise(weight = n()) %>%
  filter(from!=to) %>%
  filter(weight > 1) %>%
  ungroup()

27.8.2 Plotting the first interactive network graph

The code chunk below will be used to plot an interactive network graph by using the data prepared.

visNetwork(GAStech_nodes, 
           GAStech_edges_aggregated)

---

visNetwork(GAStech_nodes,
           GAStech_edges_aggregated) %>%
  visIgraphLayout(layout = "layout_with_fr") 

---

Visit Igraph to find out more about visIgraphLayout’s argument.

27.8.4 Working with visual attributes - Nodes

visNetwork() looks for a field called “group” in the nodes object and colour the nodes according to the values of the group field.

The code chunk below rename Department field to group.

Show the code

When we rerun the code chunk below, visNetwork shades the nodes by assigning unique colour to each category in the group field.

visNetwork(GAStech_nodes,
           GAStech_edges_aggregated) %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visLegend() %>%
  visLayout(randomSeed = 123)

---

27.8.5 Working with visual attributes - Edges

In the code run below visEdges() is used to symbolise the edges.
- The argument arrows is used to define where to place the arrow.
- The smooth argument is used to plot the edges using a smooth curve.

visNetwork(GAStech_nodes,
           GAStech_edges_aggregated) %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visEdges(arrows = "to", 
           smooth = list(enabled = TRUE, 
                         type = "curvedCW")) %>%
  visLegend() %>%
  visLayout(randomSeed = 123)

---

GAStech_nodes$group <- GAStech_nodes$Department


visNetwork(GAStech_nodes, GAStech_edges_aggregated) %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visEdges(arrows = "to", 
           smooth = list(enabled = TRUE, type = "curvedCW")) %>%
  visLegend(useGroups = TRUE, position = "right") %>%
  visLayout(randomSeed = 123)

---

Visit Option to find out more about visEdges’s argument.

27.8.6 Interactivity

In the code chunk below, visOptions() is used to incorporate interactivity features in the data visualisation.

• The argument highlightNearest highlights nearest when clicking a node.

• The argument nodesIdSelection adds an id node selection creating an HTML select element.

visNetwork(GAStech_nodes,
           GAStech_edges_aggregated) %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visOptions(highlightNearest = TRUE,
             nodesIdSelection = TRUE) %>%
  visLegend() %>%
  visLayout(randomSeed = 123)

Select by idMat.BramarAnda.RiberaRachel.PantanalLinda.LagosRuscella.Mies.HaberCarla.ForluniauCornelia.LaisKanon.HerreroVarja.LagosStenig.FusilMarin.OndaAxel.CalzasBrand.TempestadElsa.OrillaIsande.BorrascaKare.OrillaFelix.BalasLars.AzadaLidelse.DedosWillem.Vasco-PaisBertrand.OvanAdra.NubarronBirgitta.FrenteGustav.CazarVira.FrenteLinnea.BergenLucas.AlcazarIsak.BazaNils.CalixtoSven.FlechaEmile.ArpaVarro.AwelonDante.CoginianEdvard.VannHennie.OsvaldoIsia.VannMinke.MiesSten.Sanjorge.JrFelix.ResumirHideki.CocinaroInga.FerroLoreto.BodrogiIngrid.BarrancoAda.Campo-CorrenteOrhan.StrumAdan.MorlunAlbina.HafonBenito.HawelonCecilia.MorluniauClaudio.HawelonDylan.ScozzeseHenk.MiesIrene.NantValeria.Morlun

---

Visit Option to find out more about visOption’s argument.

27.9 Additional Exercise

As part of Take-Home Exercise 1, we were required to reference a fellow student’s exercise. A thought of how referenced each site was then ocurred, I did some webscrapping in Python using BeautifulSoup based on the netlify links of students provided.

The reference matrix was then used to perform the below interactive network graph.

# Load packages
pacman::p_load(tidyverse, visNetwork)

# Read files
edges_raw <- read_csv("data/recursive_site_reference_matrix_14052025_2025.csv")
nodes_raw <- read_csv("data/site_names.csv", col_names = c("name", "label"))

# === 1. Process nodes: Use student name as label, short URL as ID ===
nodes <- nodes_raw %>%
  mutate(id = str_remove(label, "https://"),  # match column headers
         label = name,                        # display student name
         group = "Sites") %>%
  select(id, label, group)

# === 2. Reshape adjacency matrix to long format ===
edges <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  transmute(from = source, to = target, weight = 1)

# === 3. Draw interactive network ===
visNetwork(nodes, edges) %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visEdges(arrows = "to", smooth = list(enabled = TRUE, type = "curvedCW")) %>%
  visOptions(highlightNearest = TRUE, nodesIdSelection = TRUE) %>%
  visLegend(useGroups = TRUE, position = "right") %>%
  visLayout(randomSeed = 123)

Select by idANDRE ONG JIA KANGCALVIN TAN SONG HAOCHAN HAN JIECHIAM SHUNZHONG DAVIDDO QUYNH TRANGENRICO SEBASTIAN ENRICO SEBASTIAN DANUSANTOSOEVANGELINE OLIVIA SIDIHARTOFU YILINGOH YI FANGHAO LIUHUANG PENGXINHUANG ZIHANJESSE EMMANUEL LUCASLAU JIA YILI JIANYILI YUQUANLIAW YING TING CELINLIM KAH SIEW BELINDALIU CHIH-YUANLU LINSENLUO YUMINGMARGA THURAMIN HTET AUNGNG JIN YAONG WEE TINN SHERMAINNNGUYEN NGUYEN HANGUYEN PHUONG HOANOR HENDRA BIN ABDUL RAHMANPATRICIA TRISNOPENG HSIAO-YUANRAJESH BABU SANTIGARISANDRA JACOBSARIYANTI HARTIONOSTEFANIE FELICIATA NGUYEN THAO NGUYENTAI QIUYANTAI YU YINGTAN WENYING AUDREYTEO WEE SIANG ROYVANESSA RIADIWANG ANQIWANG SHENSIXU XINYIYANG LUYANG YAYONGYUAN YIHAOZHANG JINGHANZHANG XUERONG

---

Some Individual nodes were noted, upon inspection of the sites their root nodes i.e. index did not hyperlink their Takehome Exercise 01 correctly or fail to reference other students in their work hence causing the script to fail in taking up any references of other sites. This was not corrected as the site scrapper takes a considerable amount of time to re-run with additional parameters. Results are to be taken for reference only!

27.9.1 Grouped by Number of Outgoing references

# === 1. Process nodes: Use student name as label, short URL as ID ===
# Count outgoing references per node to color by group
reference_counts <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  count(source, name = "ref_count")

nodes <- nodes_raw %>%
  mutate(id = str_remove(label, "https://"),
         label = name) %>%
  left_join(reference_counts, by = c("id" = "source")) %>%
  mutate(ref_count = replace_na(ref_count, 0),
         group = as.character(ref_count)) %>%  # dynamic color grouping
  select(id, label, group)

# === 2. Reshape adjacency matrix to long format ===
edges <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  transmute(from = source, to = target, weight = 1)

# === 3. Draw interactive network ===
visNetwork(nodes, edges) %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visEdges(arrows = "to", smooth = list(enabled = TRUE, type = "curvedCW")) %>%
  visOptions(highlightNearest = TRUE, nodesIdSelection = TRUE) %>%
  visLegend(useGroups = TRUE, position = "right") %>%
  visLayout(randomSeed = 123)

Select by idANDRE ONG JIA KANGCALVIN TAN SONG HAOCHAN HAN JIECHIAM SHUNZHONG DAVIDDO QUYNH TRANGENRICO SEBASTIAN ENRICO SEBASTIAN DANUSANTOSOEVANGELINE OLIVIA SIDIHARTOFU YILINGOH YI FANGHAO LIUHUANG PENGXINHUANG ZIHANJESSE EMMANUEL LUCASLAU JIA YILI JIANYILI YUQUANLIAW YING TING CELINLIM KAH SIEW BELINDALIU CHIH-YUANLU LINSENLUO YUMINGMARGA THURAMIN HTET AUNGNG JIN YAONG WEE TINN SHERMAINNNGUYEN NGUYEN HANGUYEN PHUONG HOANOR HENDRA BIN ABDUL RAHMANPATRICIA TRISNOPENG HSIAO-YUANRAJESH BABU SANTIGARISANDRA JACOBSARIYANTI HARTIONOSTEFANIE FELICIATA NGUYEN THAO NGUYENTAI QIUYANTAI YU YINGTAN WENYING AUDREYTEO WEE SIANG ROYVANESSA RIADIWANG ANQIWANG SHENSIXU XINYIYANG LUYANG YAYONGYUAN YIHAOZHANG JINGHANZHANG XUERONG

---

27.9.2 Grouped by Number of Incoming references

# Load packages
pacman::p_load(tidyverse, visNetwork)

# === 1. Read data ===
edges_raw <- read_csv("data/recursive_site_reference_matrix_14052025_2025.csv")
nodes_raw <- read_csv("data/site_names.csv", col_names = c("name", "label"))

# === 2. Count incoming references (target) ===
incoming_counts <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  count(target, name = "ref_count")

# === 3. Prepare nodes: show student name, assign color group by popularity ===
nodes <- nodes_raw %>%
  mutate(id = str_remove(label, "https://"),
         label = name) %>%
  left_join(incoming_counts, by = c("id" = "target")) %>%
  mutate(ref_count = replace_na(ref_count, 0),
         group = as.character(ref_count)) %>%
  select(id, label, group)

# === 4. Prepare edges: only links with value = 1 ===
edges <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  transmute(from = source, to = target, weight = 1)

# === 5. Draw interactive graph with sorted legend ===
visNetwork(nodes, edges) %>%
  visGroups(groupname = "0", color = "green") %>%
  visGroups(groupname = "1", color = "lightskyblue") %>%
  visGroups(groupname = "2", color = "orange") %>%
  visGroups(groupname = "3", color = "violet") %>%
  visGroups(groupname = "4", color = "purple") %>%
  visGroups(groupname = "5", color = "red") %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visEdges(arrows = "to", smooth = list(enabled = TRUE, type = "curvedCW")) %>%
  visOptions(highlightNearest = TRUE, nodesIdSelection = TRUE) %>%
  visLegend(useGroups = TRUE, position = "right") %>%
  visLayout(randomSeed = 123)

Select by idANDRE ONG JIA KANGCALVIN TAN SONG HAOCHAN HAN JIECHIAM SHUNZHONG DAVIDDO QUYNH TRANGENRICO SEBASTIAN ENRICO SEBASTIAN DANUSANTOSOEVANGELINE OLIVIA SIDIHARTOFU YILINGOH YI FANGHAO LIUHUANG PENGXINHUANG ZIHANJESSE EMMANUEL LUCASLAU JIA YILI JIANYILI YUQUANLIAW YING TING CELINLIM KAH SIEW BELINDALIU CHIH-YUANLU LINSENLUO YUMINGMARGA THURAMIN HTET AUNGNG JIN YAONG WEE TINN SHERMAINNNGUYEN NGUYEN HANGUYEN PHUONG HOANOR HENDRA BIN ABDUL RAHMANPATRICIA TRISNOPENG HSIAO-YUANRAJESH BABU SANTIGARISANDRA JACOBSARIYANTI HARTIONOSTEFANIE FELICIATA NGUYEN THAO NGUYENTAI QIUYANTAI YU YINGTAN WENYING AUDREYTEO WEE SIANG ROYVANESSA RIADIWANG ANQIWANG SHENSIXU XINYIYANG LUYANG YAYONGYUAN YIHAOZHANG JINGHANZHANG XUERONG

---

# Load packages
pacman::p_load(tidyverse, visNetwork)

# === 1. Read data ===
edges_raw <- read_csv("data/recursive_site_reference_matrix_14052025_2025.csv")
nodes_raw <- read_csv("data/site_names.csv", col_names = c("name", "label"))

# === 2. Count incoming references (target) ===
incoming_counts <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  count(target, name = "ref_count")

# === 3. Prepare nodes: show student name, assign color group and size by popularity ===
nodes <- nodes_raw %>%
  mutate(id = str_remove(label, "https://"),
         label = name) %>%
  left_join(incoming_counts, by = c("id" = "target")) %>%
  mutate(
    ref_count = replace_na(ref_count, 0),
    group = as.character(ref_count),
    value = ref_count  # Node size
  ) %>%
  select(id, label, group, value)

# === 4. Prepare edges: only links with value = 1 ===
edges <- edges_raw %>%
  rename(source = 1) %>%
  pivot_longer(-source, names_to = "target", values_to = "value") %>%
  filter(value == 1 & source != target) %>%
  transmute(from = source, to = target, weight = 1)

# === 5. Draw interactive graph with node size ===
visNetwork(nodes, edges) %>%
  visGroups(groupname = "0", color = "green") %>%
  visGroups(groupname = "1", color = "lightskyblue") %>%
  visGroups(groupname = "2", color = "orange") %>%
  visGroups(groupname = "3", color = "violet") %>%
  visGroups(groupname = "4", color = "purple") %>%
  visGroups(groupname = "5", color = "red") %>%
  visIgraphLayout(layout = "layout_with_fr") %>%
  visEdges(arrows = "to", smooth = list(enabled = TRUE, type = "curvedCW")) %>%
  visOptions(highlightNearest = TRUE, nodesIdSelection = TRUE) %>%
  visLegend(useGroups = TRUE, position = "right") %>%
  visLayout(randomSeed = 123)

Select by idANDRE ONG JIA KANGCALVIN TAN SONG HAOCHAN HAN JIECHIAM SHUNZHONG DAVIDDO QUYNH TRANGENRICO SEBASTIAN ENRICO SEBASTIAN DANUSANTOSOEVANGELINE OLIVIA SIDIHARTOFU YILINGOH YI FANGHAO LIUHUANG PENGXINHUANG ZIHANJESSE EMMANUEL LUCASLAU JIA YILI JIANYILI YUQUANLIAW YING TING CELINLIM KAH SIEW BELINDALIU CHIH-YUANLU LINSENLUO YUMINGMARGA THURAMIN HTET AUNGNG JIN YAONG WEE TINN SHERMAINNNGUYEN NGUYEN HANGUYEN PHUONG HOANOR HENDRA BIN ABDUL RAHMANPATRICIA TRISNOPENG HSIAO-YUANRAJESH BABU SANTIGARISANDRA JACOBSARIYANTI HARTIONOSTEFANIE FELICIATA NGUYEN THAO NGUYENTAI QIUYANTAI YU YINGTAN WENYING AUDREYTEO WEE SIANG ROYVANESSA RIADIWANG ANQIWANG SHENSIXU XINYIYANG LUYANG YAYONGYUAN YIHAOZHANG JINGHANZHANG XUERONG

---