opensees mp
opensees mp
Dear STKO Team,
I found a problem with parallel analysis using the openseesmp solver. I partitioned the pier model into 4 and 6 domians, or even more. The more domains, the more parts of the plastic hinge region of the pier model are partitioned, and the nonlinear phase of the hysteresis curve of the pier model drops sharply. What good methods can improve the calculation efficiency and ensure the accuracy of the results?
I found a problem with parallel analysis using the openseesmp solver. I partitioned the pier model into 4 and 6 domians, or even more. The more domains, the more parts of the plastic hinge region of the pier model are partitioned, and the nonlinear phase of the hysteresis curve of the pier model drops sharply. What good methods can improve the calculation efficiency and ensure the accuracy of the results?
Re: opensees mp
This sounds strange. Could you share your model so that I can reproduce this behavior?
Re: opensees mp
Later, I ran the same model with server and desktop respectively, but the domains partitioned were different. Hysteresis curve is shown in the figure. It is found that the number of iterations in each step is different during operation.
- Attachments
-
- Snipaste_2022-12-01_10-14-19.png (1.75 MiB) Viewed 4681 times
Last edited by cqf_2232 on Wed Dec 07, 2022 1:02 pm, edited 3 times in total.
Re: opensees mp
OK I see.
The difference is not huge: it looks like the unstable part (softening) is slightly different depending on the number of processors.
It may make sense, because softening is an instability, and the DomainDecomposition (the method used by parallel analysis in OpenSeesMP) is an iterative process.
So if the convergence is not satisfied correctly, the solution may (slightly) depend on the partition scheme.
Let's try to tackle this issue step-by-step ( I have some ideas about the issue, but it's better to test them one by one ).
First issue:
Have you noticed a huge number of iterations during the Displacement-Controlled analysis?
For computational reasons (not theoretical reasons), when you do a displacement-control analysis, try to use a reference load which is a fair estimation of what would be the real load. In this way the load-multiplier (which is unknown in the Displacement-Controlled analysis is close to 1).
In your example, you are modelling in Newtons, and you used a reference load = 1 Newton. However, in your problem, you would expect a lateral load of the order of KiloNewtons.
For example, if LoadRef = 1 N -> Lambda = 60E3 to make the final load = 60 kN
Try to use instead a LoadRef = 1e3 N (1 kN). Lambda will be 60 to have the final load = 60 kN.
Try this with just 2 cases (Server 1 Part. and Server 4 Part.), and let me know if the convergence gets better, and also if the two solutions get closer to each other.
After this, we can try other things
The difference is not huge: it looks like the unstable part (softening) is slightly different depending on the number of processors.
It may make sense, because softening is an instability, and the DomainDecomposition (the method used by parallel analysis in OpenSeesMP) is an iterative process.
So if the convergence is not satisfied correctly, the solution may (slightly) depend on the partition scheme.
Let's try to tackle this issue step-by-step ( I have some ideas about the issue, but it's better to test them one by one ).
First issue:
Have you noticed a huge number of iterations during the Displacement-Controlled analysis?
For computational reasons (not theoretical reasons), when you do a displacement-control analysis, try to use a reference load which is a fair estimation of what would be the real load. In this way the load-multiplier (which is unknown in the Displacement-Controlled analysis is close to 1).
In your example, you are modelling in Newtons, and you used a reference load = 1 Newton. However, in your problem, you would expect a lateral load of the order of KiloNewtons.
For example, if LoadRef = 1 N -> Lambda = 60E3 to make the final load = 60 kN
Try to use instead a LoadRef = 1e3 N (1 kN). Lambda will be 60 to have the final load = 60 kN.
Try this with just 2 cases (Server 1 Part. and Server 4 Part.), and let me know if the convergence gets better, and also if the two solutions get closer to each other.
After this, we can try other things
Re: opensees mp
Thank you for your sincere suggestions. I changed the lateral load to 1 kN according to your suggestion and ran Server1, Server4 and Desktop1. In addition, there are models that apply 1 N and 1 kN lateral loads respectively, both of which use opensees sequential solver. At present, the number of iterations is much less, but the softening part still has a large number of iterations. What should I do next?
- Attachments
-
- Snipaste_2022-12-02_09-55-32.png (2.02 MiB) Viewed 4655 times
Re: opensees mp
Ok it seems that the DisplacementControl with a too-small initial Reference Load (1 N) was giving convergence (and accuracy) issues. In fact now, the most different result seems to be Desktop with RefLoad = 1 N.
(by the way, the fact that in the post-peak you still have a convergence issue is normal: softening is a bad guy!)
All the others seem to be pretty much aligned.
What puzzles me is that Desktop-partition-1 is almost identical to Server-partition-4, while Server-partition-1 and Server-partition-4 have some differences.
I would have expected Server-partition-1 equal to Desktop-partition-1... Are you sure you are using the same opensees versions?
Anyway now it seems much better.
Next step:
Another source of the issue can be the IMPL-EX algorithm: It improves convergence by doing an explicit extrapolation. This introduces integration errors. Those errors diminish with smaller time steps.
So, let's try to reduce the time-step to see what happens
(by the way, the fact that in the post-peak you still have a convergence issue is normal: softening is a bad guy!)
All the others seem to be pretty much aligned.
What puzzles me is that Desktop-partition-1 is almost identical to Server-partition-4, while Server-partition-1 and Server-partition-4 have some differences.
I would have expected Server-partition-1 equal to Desktop-partition-1... Are you sure you are using the same opensees versions?
Anyway now it seems much better.
Next step:
Another source of the issue can be the IMPL-EX algorithm: It improves convergence by doing an explicit extrapolation. This introduces integration errors. Those errors diminish with smaller time steps.
So, let's try to reduce the time-step to see what happens
Re: opensees mp
Thank you very much for your guidance. I reduce the time step by increasing the number of analysis step. As the time step decreases, the number of iterations decreases. As shown in the figure, 1000 analysis steps could better solve the problem of softening inconsistency on the server. As the time step decreases, the gap between desktop and server also decreases. I want to know what you think of the simulation results? Are there any other suggestions?
- Attachments
-
- Snipaste_2022-12-04_16-44-25.png (2.3 MiB) Viewed 4625 times
Re: opensees mp
They look fine now.
So the reason was the integration error introduced by the IMPL-EX algorithm. It may happen that the local IMPLEX error is different among the different processors (for example, one process may have more elements gone into nonlinear stage than other processes).
It makes sense. IMPLEX gives you a great convergence improvement and robustness, but you need small time-steps to keep this error under control.
We are going to present soon a new material model (which is an improvement over the DamageTC3D) that will have an automatic control of the IMPLEX error.
So the reason was the integration error introduced by the IMPL-EX algorithm. It may happen that the local IMPLEX error is different among the different processors (for example, one process may have more elements gone into nonlinear stage than other processes).
It makes sense. IMPLEX gives you a great convergence improvement and robustness, but you need small time-steps to keep this error under control.
We are going to present soon a new material model (which is an improvement over the DamageTC3D) that will have an automatic control of the IMPLEX error.
Re: opensees mp
Thank you for your help and look forward to the new material model.
Re: opensees mp
The material model has been added to OpenSees:
https://opensees.github.io/OpenSeesDocu ... ete3D.html
By the end of this we will release a new STKO version (3.2.0) and you will be able to download the latest OpenSees-Solvers from prof.Tarque's website with this new material.
When you will receive the notification on STKO about the new version, please download it and download also the OpenSees-Solvers.
In January we'll do a webinar on this new material
https://opensees.github.io/OpenSeesDocu ... ete3D.html
By the end of this we will release a new STKO version (3.2.0) and you will be able to download the latest OpenSees-Solvers from prof.Tarque's website with this new material.
When you will receive the notification on STKO about the new version, please download it and download also the OpenSees-Solvers.
In January we'll do a webinar on this new material