R software stuck when running ergm

[Ru-Zheng]

The network I built is 120 nodes and 1650 edges. When I execute the command: "fit<- ergm(model ~ edges + triangle)", it has been stuck in the iteration 3~7 times during the iteration process, and does not enter the next iteration for a long time. How can I solve this problem?

[ngmaclaren]

Can I upvote this? I have similar-sounding problems, but it seems to be network dependent. I can run the above ERGM on a GNM random graph, but a seemingly similar model on a smaller network with many isolated nodes can fail (see below). Do you have isolated nodes or a very heterogeneous degree distribution? I have worked around this problem before using the gwesp term, below, in case that helps.

library(network)
library(sna)
library(ergm)

set.seed(12345)
g <- as.network(rgnm(n = 1, nv = 120, m = 1650, mode = "graph"), directed = FALSE)
summary(g ~ edges + triangle)
ergm(g ~ edges + triangle) # runs

## Different network but similar-sounding problem
h <- as.network(rgraph(35, m = 1, tprob = 0.06, mode = "digraph"))
summary(h ~ edges + mutual + transitive)
ergm(h ~ edges + mutual + transitive) # runs
summary(h ~ edges + mutual + gwesp)
ergm(h ~ edges + mutual + gwesp) # runs but does not converge in 60 iterations
ergm(h ~ edges + mutual + gwesp(0.1, fixed = TRUE)) # runs

add.vertices(h, nv = 11) # adding fewer nodes may not reproduce the problem
summary(h ~ edges + mutual + transitive)
ergm(h ~ edges + mutual + transitive) # hangs
ergm(h ~ edges + mutual + gwesp) # "Process R killed..." at iteration 55
ergm(h ~ edges + mutual + gwesp(0.1, fixed = TRUE)) # runs

[Ru-Zheng]

我可以投票吗？我有类似的问题，但它似乎与网络有关。我可以在 GNM 随机图上运行上述 ERGM，但在具有许多孤立节点的较小网络上看似相似的模型可能会失败（见下文）。您是否有孤立的节点或非常异构的度分布？在使用下面的 gwesp 术语之前，我已经解决了这个问题，以防万一。

library(network)
library(sna)
library(ergm)

set.seed(12345)
g <- as.network(rgnm(n = 1, nv = 120, m = 1650, mode = "graph"), directed = FALSE)
summary(g ~ edges + triangle)
ergm(g ~ edges + triangle) # runs

## Different network but similar-sounding problem
h <- as.network(rgraph(35, m = 1, tprob = 0.06, mode = "digraph"))
summary(h ~ edges + mutual + transitive)
ergm(h ~ edges + mutual + transitive) # runs
summary(h ~ edges + mutual + gwesp)
ergm(h ~ edges + mutual + gwesp) # runs but does not converge in 60 iterations
ergm(h ~ edges + mutual + gwesp(0.1, fixed = TRUE)) # runs

add.vertices(h, nv = 11) # adding fewer nodes may not reproduce the problem
summary(h ~ edges + mutual + transitive)
ergm(h ~ edges + mutual + transitive) # hangs
ergm(h ~ edges + mutual + gwesp) # "Process R killed..." at iteration 55
ergm(h ~ edges + mutual + gwesp(0.1, fixed = TRUE)) # runs

Thank you for your reply. YEAH! your codes give me some advice ,I will check my commands

[CarterButts]

You are using an edge-triangle model. This model is quite famously degenerate; as a result, the model is failing to converge. The hanging you are experience is an indicator of this non-convergence.

The fix in this case is to be more careful of the models you are using, and to ensure that your specification is appropriate; there is no way around reading the literature on this, but running pilot simulations at the MPLE can often give you a pretty good indication of whether the model is going to be stable or not. There are other useful schemes for diagnosing a slowly mixing model in the statnet ergm tutorials on the statnet web site, so I would also check those out.

Finally, I will also note that the current convergence detection scheme is in my experience a bit finicky, and you may need to work around it. Using the "Hotelling" convergence criterion in the control.ergm argument reverts to the method used before ergm 4.0, and in my experience this is more robust. (The new method seems to be faster/surer when it works, but it sometimes goes off the rails.) A more radical approach is to use the stochastic approximation method (see the main.method argument to control.ergm), which needs (in my experience) more tweaking to produce really high-quality estimates, but which can be much faster/able to give reasonable approximations with difficult models.

Also, you should probably direct questions regarding model fitting to the statnet help list, since this is for bug reports. This doesn't seem to be a bug, just the usual behavior of that model.

[CarterButts]

Seems not to be a bug, just slow convergence with a known degenerate model. But it would be good to improve convergence with the new criterion, regardless, and perhaps provide more explicit tracking messages.

[Ru-Zheng]

You are using an edge-triangle model. This model is quite famously degenerate; as a result, the model is failing to converge. The hanging you are experience is an indicator of this non-convergence.

The fix in this case is to be more careful of the models you are using, and to ensure that your specification is appropriate; there is no way around reading the literature on this, but running pilot simulations at the MPLE can often give you a pretty good indication of whether the model is going to be stable or not. There are other useful schemes for diagnosing a slowly mixing model in the statnet ergm tutorials on the statnet web site, so I would also check those out.

Finally, I will also note that the current convergence detection scheme is in my experience a bit finicky, and you may need to work around it. Using the "Hotelling" convergence criterion in the control.ergm argument reverts to the method used before ergm 4.0, and in my experience this is more robust. (The new method seems to be faster/surer when it works, but it sometimes goes off the rails.) A more radical approach is to use the stochastic approximation method (see the main.method argument to control.ergm), which needs (in my experience) more tweaking to produce really high-quality estimates, but which can be much faster/able to give reasonable approximations with difficult models.

Also, you should probably direct questions regarding model fitting to the statnet help list, since this is for bug reports. This doesn't seem to be a bug, just the usual behavior of that model.

ok , thank you very much, your suggestions are so great, it's help me a lot , I will read the tutorials right now and remember all of your tips!

[ngmaclaren]

If you don't already have it, this book has been very helpful for me and was where I found the gwesp(0.1, fixed = TRUE) solution to the convergence problems I was experiencing:

Harris, J. K. (2013). An introduction to exponential random graph modeling (Vol. 173). Sage Publications.

[Ru-Zheng]

Thank you!very helpful