---
title: "Who wrote the Federalist papers?"
author: "Savaş Dayanık"
date: '2022-03-24'
output: 
  html_document:
    toc: yes
    code_folding: "hide"
---

```{r setup, include=FALSE}
library(magrittr)
library(tidyverse)
library(pander)
library(car)
library(rstan)
library(class)
knitr::opts_chunk$set(echo = TRUE)
```


# Introduction

[Wikipedia on The Federalist Papers](https://en.wikipedia.org/wiki/The_Federalist_Papers)

> The Federalist Papers is a collection of 85 articles and essays written by Alexander Hamilton, James Madison, and John Jay under the collective pseudonym "Publius" to promote the ratification of the United States Constitution. The collection was commonly known as The Federalist until the name The Federalist Papers emerged in the 20th century. They are published between October 27, 1787 and May 28, 1788.

[Wikipedia on Origins](https://en.wikipedia.org/wiki/The_Federalist_Papers#Origins)

> The Federal Convention (Constitutional Convention) sent the proposed Constitution to the Confederation Congress, which in turn submitted it to the states for ratification at the end of September 1787. On September 27, 1787, "Cato" first appeared in the New York press criticizing the proposition; "Brutus" followed on October 18, 1787. These and other articles and public letters critical of the new Constitution would eventually become known as the "Anti-Federalist Papers". In response, Alexander Hamilton decided to launch a measured defense and extensive explanation of the proposed Constitution to the people of the state of New York. He wrote in Federalist No. 1 that the series would "endeavor to give a satisfactory answer to all the objections which shall have made their appearance, that may seem to have any claim to your attention."

```{r, out.width=c("50%", "48.5%"), fig.show="hold"}
knitr::include_graphics("figures/titlepage.jpg")
knitr::include_graphics("figures/backpage.jpg")
```


> Hamilton recruited collaborators for the project. He enlisted John Jay, who after four strong essays (Federalist Nos. 2, 3, 4, and 5), fell ill and contributed only one more essay, Federalist No. 64, to the series. Jay also distilled his case into a pamphlet in the spring of 1788, An Address to the People of the State of New-York; Hamilton cited it approvingly in Federalist No. 85. James Madison, present in New York as a Virginia delegate to the Confederation Congress, was recruited by Hamilton and Jay and became Hamilton's primary collaborator. 


[Wikipedia on Authorship](https://en.wikipedia.org/wiki/The_Federalist_Papers#Authorship)

> **Authorship**
>
> James Madison, Hamilton's major collaborator, later fourth President of the United States (1809-1817)
>
> At the time of publication, the authors of The Federalist Papers attempted to hide their identities due to Hamilton and Madison having attended the convention. Astute observers, however, correctly discerned the identities of Hamilton, Madison, and Jay. Establishing authorial authenticity of the essays that constitute The Federalist Papers has not always been clear. After Alexander Hamilton died in 1804, a list emerged, claiming that he alone had written two-thirds of The Federalist essays. Some believe that several of these essays were written by James Madison (Nos. 49–58 and 62–63). The scholarly detective work of Douglass Adair in 1944 postulated the following assignments of authorship, corroborated in 1964 by a computer analysis of the text:
>
> * Alexander Hamilton (51 articles: Nos. 1, 6–9, 11–13, 15–17, 21–36, 59–61, and 65–85)
> * James Madison (29 articles: Nos. 10, 14, 18–20, 37–58 and 62–63)
> * John Jay (5 articles: Nos. 2–5 and 64).
>
> In six months, a total of 85 articles were written by the three men. Hamilton, who had been a leading advocate of national constitutional reform throughout the 1780s and was one of the three representatives for New York at the Constitutional Convention, in 1789 became the first Secretary of the Treasury, a post he held until his resignation in 1795. Madison, who is now acknowledged as the father of the Constitution—despite his repeated rejection of this honor during his lifetime, became a leading member of the U.S. House of Representatives from Virginia (1789–1797), Secretary of State (1801–1809), and ultimately the fourth President of the United States (1809–1817). John Jay, who had been secretary for foreign affairs under the Articles of Confederation from 1784 through their expiration in 1789, became the first Chief Justice of the United States in 1789, stepping down in 1795 to accept election as governor of New York, a post he held for two terms, retiring in 1801. 


[Wikipedia on Disputed essays ](https://en.wikipedia.org/wiki/The_Federalist_Papers#Authorship) says

> **Disputed essays**
>
> While the authorship of 73 of The Federalist essays is fairly certain, the identities of those who wrote the twelve remaining essays are disputed by some scholars. The modern consensus is that Madison wrote essays Nos. 49–58, with Nos. 18–20 being products of a collaboration between him and Hamilton; No. 64 was by John Jay. The first open designation of which essay belonged to whom was provided by Hamilton who, in the days before his ultimately fatal gun duel with Aaron Burr, provided his lawyer with a list detailing the author of each number. This list credited Hamilton with a full 63 of the essays (three of those being jointly written with Madison), almost three-quarters of the whole, and was used as the basis for an 1810 printing that was the first to make specific attribution for the essays.
>
> Madison did not immediately dispute Hamilton's list, but provided his own list for the 1818 Gideon edition of The Federalist. Madison claimed 29 essays for himself, and he suggested that the difference between the two lists was "owing doubtless to the hurry in which [Hamilton's] memorandum was made out." A known error in Hamilton's list — Hamilton incorrectly ascribed No. 54 to John Jay, when in fact, Jay wrote No. 64 — provided some evidence for Madison's suggestion.
>
> Statistical analysis has been undertaken on several occasions in attempts to accurately identify the author of each individual essay. After examining word choice and writing style, studies generally agree that the disputed essays were written by James Madison. However, there are notable exceptions maintaining that some of the essays which are now widely attributed to Madison were, in fact, collaborative efforts.

Similar questions arised for the works of Shakespeare, Homer.


```{r}
d <- ProbBayes::federalist_word_study %>% 
  mutate_if(is.factor, as.character) %>% 
  mutate(Authorship = ifelse(Authorship == "", "Jay", Authorship),
         Authorship = ifelse(Disputed == "yes", "Hamilton or Madison", Authorship))

d %>% 
  distinct(Name, Authorship, Disputed) %>% 
  count(Authorship, Disputed) %>% 
  pander()

panderOptions("table.alignment.default", c("left", "left"))

d %>% 
  distinct(Name, Authorship, Disputed) %>% 
  mutate(Name = Name %>% str_replace("\\.", "") %>% parse_number()) %>% 
  arrange(Authorship, Name) %>% 
  group_by(Authorship) %>% 
  summarize(Name = paste(Name, collapse = ", ")) %>% 
  pander()
```

* Mosteller and Wallace (1963) illustrated how to use Bayesian reasoning in solving authorship problem.


```{r, out.width=c("50%", "44.7%"), fig.show = "hold"}
knitr::include_graphics("figures/mosteller-wallace.jpg")
knitr::include_graphics("figures/mosteller-wallace-recent.jpg")
```

* [Applied Bayesian and Classical InferenceThe Case of The Federalist Papers](https://link.springer.com/book/10.1007/978-1-4612-5256-6)
* [Gutenberg project](https://www.gutenberg.org/ebooks/18)
* [Shakespeare authorship question](https://en.wikipedia.org/wiki/Shakespeare_authorship_question)
* [Homeric Question](https://en.wikipedia.org/wiki/Homeric_Question)

# Data on word use

* Focus on frequencies of word counts (across different papers, normalized, say in every 1,000 words)
* Focus on "filler" words (also called stopwords)
  - Examples
    - an
    - of
    - upon
    - can
  - Why fillers? 
    - Because others will depend on the topic of the current topic as well.
    - Because fillers are expected to appear at the same rate across all articles, but the rate will depend on the style of the author. Well at least for some filler words -- we hope.
* If you want to learn topics of papers, not the author styles, then remove stopwords and study the co-occurrences of the remaining contextual words.
    
```{r, out.width="120%"}
d %>% 
  filter(Disputed == "no", Authorship != "Jay") %>% 
  filter( word %in% c("the", "of", "to", "and", "be", "that")) %>% 
  ggplot(aes(Rate)) +
  geom_density(aes(fill=Authorship), alpha=0.5) +
  facet_wrap(~word, scale = "free_x")
```
  
```{r, fig.asp=1, out.width = "110%"}
pairwise <- d %>% 
  filter(Disputed == "no", Authorship != "Jay") %>% 
  filter( word %in% c("the", "of", "to", "and", "be", "that")) %>% 
  select(Name, word, Rate, Authorship) %>% 
  pivot_wider(names_from = word, values_from = "Rate") %>% 
  select(-Name)

scatterplotMatrix( ~ the + of + and + be + that | Authorship, data = pairwise, 
                   smooth = FALSE, regLine = FALSE, ellipse = list(levels = 0.90))  
```

Can be used non-Bayesian methods like
* Use Fisher discriminant analysis (FDA)
* kNN

# Classification with kNN

```{r}
all <- d %>% 
  select(Name, word, Rate, Authorship) %>% 
  pivot_wider(names_from = word, values_from = Rate) %>% 
  mutate_if(is.numeric, ~ifelse(is.na(.), 0, .)) %>% 
  select(-Name)

train <- all %>% 
  filter(Authorship %in% c("Hamilton", "Madison"))

res_pc <- prcomp(train[,-1])
res_pc %>% plot()
(res_pc$sdev)^2 %>% {./sum(.)} %>% cumsum %>% plot()

numPC <- 68
xtrain <- res_pc$x[,seq(numPC)]

```

```{r}
acc <- NULL
krange <- 1:10
for (i in krange) {
  res_knn <- knn.cv(train = xtrain, cl = train$Authorship, k = i)
  acc <- c(acc, table(res_knn, train$Authorship) %>% prop.table() %>% diag %>% sum()) 
  # cat(i, acc, "\n")
} # if there is tie between the counts of two counts, then tie is broken at random.
plot(krange, acc, t="b")
```

```{r}
test <- all %>% 
  filter(Authorship %in% c("Hamilton or Madison"))

xtest <- predict(res_pc, test)[, seq(numPC)]
pred <- knn(train=xtrain, test=xtest, cl = train$Authorship, k = which.max(acc))
table(pred)
```




