create pilot-v3

experiments/pilot-indirectSource-V3/CoverStoriesHW/Background.txt

@@ -0,0 +1,3 @@
+<p> <strong>Background:</strong> We are now at an age of great exploration and discovery. Humanity has reached the stars, venturing to distant planets in search of new frontiers and resources. One of these new worlds is a remote, resource-rich planet called <strong>Xelifan-3</strong></p>
+          <p>The year is 2184, and Xelifan-3 has become home to a small but determined group of colonists. Their survival depends on a rare plant known as <strong>Xeliherb</strong>, a life-sustaining resource native to the planet. This unique plant has extraordinary properties, providing essential nutrients necessary for survival on the planet. Furthermore, it holds potential as a key ingredient in medicines capable of curing diseases, which has attracted significant interest from scientists around the galaxy. </p> 
+          <p>However, xeliherb grows only <strong>sparsely</strong>, and its limited availability has made life on Xelifan-3 a constant struggle.</p>

experiments/pilot-indirectSource-V3/CoverStoriesHW/RoleColonist.txt

@@ -0,0 +1,9 @@
+You are one of the <strong>colonists</strong>, part of the group that has risked everything to settle on Xelifan-3 in search of a new beginning.
+<br><br>
+The colony’s survival is entirely dependent on a steady supply of <strong>Xeliherb</strong>, but its scarcity has led to a sense of urgency and tension. You live on the frontlines, working closely with the land, and every discovery you make has an immediate impact on the well-being of the people around you.
+<br><br>
+As a colonist, your mission is not just about survival but about the future of humanity’s presence on Xelifan-3. You must work alongside explorers and scientists to identify new Xeliherb patches, navigate the dangerous environments of the planet, and ensure the plant is yield and harvested efficiently.
+<br><br>
+However, there can be tension between the colonists and scientists. While the colonists focus on immediate survival, the scientists are driven by long-term research goals, which can lead to differing perspectives. Each group’s unique experiences and priorities may influence how they view and interpret information about <strong>Xeliherb</strong>.
+<br><br>
+<strong>Will you trust the scientists’ distant analysis, or will you rely on your own experiences and the stories shared by other colonists to guide your decisions?</strong>

experiments/pilot-indirectSource-V3/CoverStoriesHW/RoleScientist.txt

@@ -0,0 +1,6 @@
+<p>You are one of the <strong>scientist</strong>, part of an elite team tasked with uncovering the secrets of <strong>Xeliherb.</strong> The Xeliherb is not only a vital resource for the colony's survival, but it also represents a groundbreaking opportunity for scientific advancement. If its full potential can be unlocked, it could revolutionize medicine and become a powerful asset to humanity, well beyond the confines of Xelifan-3.
+            </p>
+            <p>Unlike the colonists, your mission is driven by discovery and innovation. You are going to conduct thorough research on Xeliherb, understanding its properties and developing new methods to maximize its yield. From your remote research stations or safe compounds, you receive data from the colonists and analyze the Xeliherb samples they send you. You must carefully interpret this information, using your scientific expertise to provide guidance on how to cultivate and harvest the plant more effectively.
+<br><br>However, there can be tension between the colonists and scientists. While the colonists focus on immediate survival, the scientists are driven by long-term research goals, which can lead to differing perspectives. Each group’s unique experiences and priorities may influence how they view and interpret information about <strong>Xeliherb</strong>.
+<br><br> <strong> Will you rely on the data and scientific methods you trust, or will you consider the colonists' on-the-ground experiences when advising on how to proceed?</strong>
+            </p>

experiments/pilot-indirectSource-V3/CoverStoriesHW/speakerJournal.txt

@@ -0,0 +1,9 @@
+Now, it is time to make a decision.
+<br><br>
+After months of surviving the hostile environment of Xelifan-3, you and your fellow collegues receive a mysterious discovery. One evening, a weathered journal is found in the ruins of an old research outpost, a relic from a science team that explored the planet decades ago. The journal is fragile and faded, but its contents are shocking. It claims:
+<br><br>
+<strong>The presence of Xeliherb is associatied with another plant called Ralocrop.</strong>
+<br><br>
+Do you trust the words of a lost science team, written in a time long past, and act on this information? Or do you continue relying on the methods that have kept the colony alive so far, unsure if the connection between xeliherb and ralocrop is real?
+<br><br>
+<strong>Question: How likely are going to looking for more Ralocrop? Your decision will shape the colony’s future.</strong> 

experiments/pilot-indirectSource-V3/CoverStoriesHW/speakerScientist.txt

@@ -0,0 +1,9 @@
+Now, it is time to make a decision.
+<br><br>
+While navigating the challenges of life on Xelifan-3, a breakthrough occurs. A member of your supporting science team, stationed at a remote research facility, shares an urgent discovery. During their latest field study, they found strong evidence suggesting:
+<br><br>
+<strong>The presence of Xeliherb is associatied with another plant called Ralocrop.</strong>
+<br><br>
+Do you trust the direct word of this scientist and their data? Or do you remain cautious, knowing that a failed experiment could threaten the colony’s delicate balance?
+<br><br>
+<strong>Question: How likely are going to looking for more Ralocrop? Your decision will shape the colony’s future.</strong> 

experiments/pilot-indirectSource-V3/README.md

@@ -0,0 +1,4 @@
+# Pilot 'indirect sources'
+
+
+Live version of the experiment [here](https://magpie-ea.github.io/magpie3-causal-implicature/experiments/pilot-indirectSource/).

experiments/pilot-indirectSource-V3/analysis/analysis-script-pilot-v2.Rmd

@@ -0,0 +1,171 @@
+---
+title: "Analysis Script Pilot Indirect Source v1"
+author: "Hening Wang"
+date: "2024-11-18"
+output: pdf_document
+editor_options: 
+  markdown: 
+    wrap: 72
+---
+Supress messages in this document so that the output is cleaner
+```{r setup, include=FALSE} 
+knitr::opts_chunk$set(warning = FALSE, message = FALSE) 
+```
+Import packages.
+```{r}
+rm(list = ls())
+
+options(warn = -1)
+library(tidyverse)
+library(lme4)
+library(lmerTest)
+library(ggplot2)
+library(aida)
+```
+
+Set up theme for ggplot.
+
+```{r}
+theme_set(theme_aida())
+
+##################################################
+## CSP-colors
+##################################################
+CSP_colors = c(
+  "#7581B3", "#99C2C2", "#C65353", "#E2BA78", "#5C7457", "#575463",
+  "#B0B7D4", "#66A3A3", "#DB9494", "#D49735", "#9BB096", "#D4D3D9",
+  "#414C76", "#993333"
+  )
+# setting theme colors globally
+scale_colour_discrete <- function(...) {
+  scale_colour_manual(..., values = CSP_colors)
+}
+scale_fill_discrete <- function(...) {
+  scale_fill_manual(..., values = CSP_colors)
+}
+```
+
+# Data Import & Preprocessing
+
+Import data
+
+```{r}
+data <- read.csv("../data/results.csv")
+```
+
+Merge decision1 and decision2 into one row by the same submission id
+```{r}
+data <- data %>% group_by(submission_id, informationSource) %>% 
+  summarize(
+    decision1 = first(na.omit(decision1)),
+    decision2 = first(na.omit(decision2))
+  ) %>%
+  mutate(update = decision2 - decision1)
+
+```
+
+Factorise variables: id, item, informationSource, listenerRole
+
+```{r}
+data$id <- as.factor(1:nrow(data))
+data$informationSource <- as.factor(data$informationSource)
+```
+
+.
+
+
+Recruitment criteria: 1. English native speakers 2. Approval rate \> 90%
+3. Completion \> 5 times 4. Not included in the prior studies of
+CommuniCause
+
+# Plots
+
+## A histogram showing the distribution of probs. 
+**Interpretation: Data is left-skewed.**
+
+```{r}
+ggplot(data, aes(x = update)) +
+  geom_histogram(binwidth = 5, fill = CSP_colors[1]) +
+  labs(title = "Distribution of updated probs",
+       x = "Updated probs.",
+       y = "Frequency")
+
+```
+
+## Plot the dependent variable "probs" against conditions.
+
+**Interpretation:**
+
+Hi Michael,
+
+I've conducted a pilot with our new design with 40 pps. The sample size is unbalanced: N(direct) = 23, N(indirect) = 17. The results are as follows:
+
+1. The updated probs are higher than 0 in both conditions (p < 0.001). Interpretation: This indicates that participants used the new association sentence as supporting evidence to strengthen their initial belief that intervening on a cause would result in a greater effect.
+
+2. The updated probs are higher in the direct condition than in the indirect condition. A one slide t-test is marginally significant (p = 0.06). Interpretation: This suggests that participants placed greater trust in the direct information source when updating their beliefs, likely because the causal link was explicitly established.
+
+For visualization of 1 & 2, see Fig. 1.
+
+3. When it comes to raw probs (see Fig. 2), the indirect condition has a higher probs than the direct condition at the initial decision (p = 0.04). Interpretation: This suggests that, in the absence of further information, the implicit causal link provided by the science team for localization narrative could in turn more likely to elicit a causal interpretation of the association sentence than a science team with a much more clear and aligned goal. This is kind of contrast intuition, however, in line with our idea that the localization team could somehow makes the causal link more salient to pps. A reason that I only have a gut feeling but cannot verbalize yet. I think we need to dive deeper into this (see below). 
+
+After analyzing and interpreting these results I have few more thoughts about our old 2x2 design. 
+1. I think the idea about updating is very nice. It reveals more information about participant's belief. 
+2. The direct/indirect manipulation we now have could break down to manipulation in both goals and sources. The source part is straight forward. It relates to the uncertainty of the information source. The goal part could relate to the alignment of speaker. In our current design, due to our question there is a discrepancy in alignment in the indirect condition: the science team's goal is localization, the participant's goal is cultivation. Finally, the difference between localization and cultivation narrative could relate to different degrees of causality (experimenting vs. observing vs. predicting).
+
+In other words, the current indirect condition has changed in more than one aspect than just indirect/direct source across screens. We might want to control these aspects more carefully with more factors, resulting in a more complex design.
+
+Could we perhaps arrange a meeting at your convenience to discuss these results and possible next steps?
+
+
+
+```{r}
+long_data <- data %>%
+  pivot_longer(cols = starts_with("decision"),
+               names_to = "decisionType",
+               values_to = "probs",
+               values_drop_na = TRUE)
+
+p1 <- ggplot(long_data, aes(x = informationSource, y = probs, fill = decisionType)) +
+  geom_boxplot() +
+  labs(title = "Probs. by information source and decision \n type",
+       x = "Information Source",
+       y = "Probs.")
+
+ggsave("fig2.png", p1, width = 6, height = 4, dpi = 300)
+```
+
+
+
+```{r}
+p2 <- ggplot(data, aes(x = informationSource, y = update)) +
+  geom_boxplot(fill = CSP_colors[1]) + 
+  labs(title = "Updated probs by information source",
+       x = "Information Source",
+       y = "Updated probs.")
+
+ggsave("fig1.png", p2, width = 6, height = 4, dpi = 300)
+```
+
+# Statistical Analysis
+**Note: Nothing is significant here.** 
+```{r}
+data_direct <- data$update[data$informationSource == "direct"]
+data_indirect <- data$update[data$informationSource == "indirect"]
+t.test(data_direct, data_indirect, alternative = "greater")
+```
+
+```{r}
+t.test(data_indirect, mu = 0, alternative = "greater")
+t.test(data_direct, mu = 0, alternative = "greater")
+```
+
+```{r}
+lm(probs ~ informationSource * decisionType, data = long_data) %>% summary()
+```
+```{r}
+data_decision1_direct <- data$decision1[data$informationSource == "direct"]
+data_decision1_indirect <- data$decision1[data$informationSource == "indirect"]
+t.test(data_decision1_direct, data_decision1_indirect, alternative = "less")
+```
+
+

experiments/pilot-indirectSource-V3/analysis/analysis.Rproj

@@ -0,0 +1,13 @@
+Version: 1.0
+
+RestoreWorkspace: Default
+SaveWorkspace: Default
+AlwaysSaveHistory: Default
+
+EnableCodeIndexing: Yes
+UseSpacesForTab: Yes
+NumSpacesForTab: 2
+Encoding: UTF-8
+
+RnwWeave: Sweave
+LaTeX: pdfLaTeX