What are common Mathematical Programming Languages out there?

Question

I've seen the term used Mathematical Programming to describe a superset of:

1. Linear programming
2. Quadratic programming
3. Nonlinear programming
4. Mixed-integer programming
5. Mixed-integer nonlinear programming
6. Second-order cone programming
7. Global optimization
8. Constraint programming

I'm looking for a list of programming languages that your can use for solving such problems.

NOTE:

1. Although it is true that the word "programming" as used in optimization doesn't mean "computer programming". And, languages like Matlab, python, C++, Julia, etc, can be used to solve optimization problems
2. What I'm looking for a language that allows the easy specification of Math Programming problems and may (Or may-not) use a Matlab, python, C++, Julia, etc. backend for the actual solution

Answer 1

Some Math Programming modeling languages closest to "math" are AMPL, GAMS, AIMMS, Cplex/OPL, Minizinc. These are specialized languages to express math programming models. Working in these languages is sometimes called "modeling" instead of "programming". Note that these languages also often support Nonlinear Programming solvers (and they may provide services like automatic differentiation -- very useful in large complex models).

Examples of systems somewhere in between modeling and programming are Gurobi/Python, Cplex/Concert, Pyomo, Pulp, Julia/JuMP. They are typically Python or c++ libraries but using some clever Object Oriented language constructs, you can express the models reasonably painlessly. In general not as compact as the specialized languages, but in return you get a more familiar programming environment.

At the low end are solver APIs, e.g. in C. This may require to build matrices and do quite some gymnastics before you can pass on your model to the solver. This is for the hard-core programmers and is usually not a good approach for very complex models. For very structured models it may not be a big hurdle.

Somewhere between the last two categories there is another group of modeling tools. Languages like R and Matlab provide a nice interface to otherwise somewhat low-level solver APIs. Building up a matrix in these languages is somewhat easier than in C but you don't really write equations.

Finally it is always possible to write MPS or LP files from any programming environment and pass those on to the solver.

Answer 2

Matematical "programming" doesn't mean the same thing as computer programming. Programming in this context simply means optimization.

A good choice that is pretty simple to learn, mainly because its syntax very closely resembles mathematical notation, is AMPL. It is built to model exactly the problems you mention. Note that AMPL is only a modeling language however - you will need additional software to actually solve the problems. These are compatible with AMPL and not hard to set up either. CPLEX is an example of a commercial solver, but there are also free solvers available, like Gurobi.

Answer 3

Bryan Birch is credited with once saying that he programmed in a very high-level programming language called "graduate student". Two of my favorites are Haskell and Ruby. I like Haskell because it is computable category theory and Ruby because it is really easy to write scripts and other various prototype kinds of code. Although any language that has functional programming constructs and closures is usually fun to use. Maple and Mathematica fall in this category and they are fun to use when I need heavy duty symbolic and plotting facilities. Also, Mathematica and Matlab are popular among mathematicians. R is popular among statisticians. But most of All One language I still use is PostScript. I probably need to defend that.

Its syntax is elegant. In fact, no language I've seen has more uniform syntax: a complete program is syntactically identical to almost any fragment of a program. There are no keywords and very few special cases.

It can be a lot of fun, and you can make pretty pictures.

It has very few data types, but some of the ones it does have are surprisingly useful. Dictionaries come to mind immediately. Also, "arrays" (which would be called "lists" in any other language) are extremely flexible. They automatically support comprehensions, not as a separate feature, but as an obvious consequence of the syntax. Functional programmers shouldn't be surprised that procedures are useful as a type; actually, due to the simplicity of the syntax and lack of keywords, any nontrivial program has to work with procedures as data.

Unfortunately, it has many drawbacks that prevent it from really being useful. Its handling of strings is abominable. Also, it has no facilities for user interaction. Its console I/O is crippled. Things like that could in principle be fixed by appropriate third-party packages, but unfortunately, to my knowledge, there are no third-party packages at all (at least for general programming). Finally, it can be very hard to debug; actually, it is more difficult to debug than any other language I know except assembly. All of those things combine to make it one of the most programmer-unfriendly languages out there. Nevertheless, some of my best work is implemented directly in PostScript, and I have done some real work in it. (Also, let's be fair: PostScript was never intended for general-purpose programming! Using a page-description language for any serious computation at all is some sort of achievement.)

For real mathematical figures (such as for inclusion in papers), I use MetaPost. PostScript can be used for this purpose, but MetaPost is much better suited for this and is very TeX-friendly.

Another language that I use mostly for fun, not serious work, is x86 assembly language. In contrast to PostScript, it's an ugly language, but strangely, I think I use assembly for some of the same reasons that draw me to PostScript.

The rest of the languages I use need no introduction: C, C++, Python, Ruby, Java.

(Languages: C, C++, Python, Ruby, Java. Mathematical interest: none in particular, but they're useful in general programming, including mathematical programs.)

I used to use Octave, but apparently most of the world uses Matlab, and Octave has just enough incompatibilities with Matlab to make it annoying to try to use other people's code. Also, it seems to have pretty poor support for sparse matrix computations.

I used to use PHP a lot. Actually, PHP and assembly are sort of an odd couple. A while ago, for no good reason, I tried to come up with the fastest code to print out all the permutations of a string. My best solution (for strings of ~10 or more characters, IIRC) was a combination of PHP and x86 assembly. To be fair, the PHP part could have been done in another language, but PHP was almost the right tool for the job.

(Language: PHP. Mathematical interest: none in particular, but it's great for designing websites with server-side scripting, which is no less useful to mathematicians than it is to other programmers.)

There are other languages I find interesting but never learned properly, like Lisp, Fortran, and Forth.