Getting Help: Find a Function Using Wildcards

There are over 4000 built-in functions in Mathematica, not including those in add-on Packages. 

In[340]:= Names["System`*"]//Length

Out[340]= 4131

Therefore it makes sense to see if there is a built-in function before we write one. Likewise, the large number of functions means that we often have a vague memory of a function we've used but can't put our finger on its name when we need it again. If I have some idea of the name of a function I've used, or want to see if there is an existing function and guess at part of its name, I use a wildcard in my current Notebook with the short form of Information (?). To find a function or survey related functions, this is faster than using the Doc Center.

Here I want to know the filepath of my current Notebook, which I cannot find in any Mathematica menu item. Maybe there's a Notebook function to tell me. I scan down the results until I see NotebookFileName, which is the one I want. I also quickly click on a half-dozen or so functions to refresh my memory of them or explore new ones and read the short info displayed below the results.

Using Information this way is always fruitful for me since I efficiently learn about functions I didn't remember or didn't know. But if it fails to flush out the function I want, I go to the Doc Center and search there.

Flat (Function Attribute of associativity; removes brackets and parentheses)

Why does this function works for more than the two arguments for which it is defined?

In[100]:= log[a_ b_]:=log@a+log@b

In[101]:= log[2 x y]

Out[101]= log[2]+log[x]+log[y]

The answer is that Times has the Attribute Flat, which is Mathematica's interesting name for the mathematical property of associativity. I.e., as we learned in school, (2 + 3) + 4 = 2 + (3 + 4). We can remove the parentheses. So the Attribute is called Flat since, after the name for the function Flatten that removes List brackets, Mathematica knows that for some functions it's valid to remove all parentheses or List brackets. What is handy is to automatically remove all brackets from a user-defined function, use SetAttributes[ function, Flat].

Thus Time's Attributes show it to be more profound than we might expect. We check function Attributes like this:

In[102]:= Attributes@Plus

Out[102]= {Flat,Listable,NumericFunction,OneIdentity,Orderless,Protected}

And we can find all built-in functions with a given attribute like this, first taking all function Names from the System context where they live, using the wildcard "*", then using a pattern in Cases:

In[103]:= Names@"System`*"//Cases[#,_?(MemberQ[Attributes@#,Orderless]&)]&

Out[103]= {ArithmeticGeometricMean,BitAnd,BitOr,BitXor,CoprimeQ,DiracDelta,DiscreteDelta,Equivalent,GCD,HeavisideTheta,KroneckerDelta,LCM,Majority,Max,Min,Multinomial,Plus,Times,UnitStep,Xnor,Xor}

Last, Trace shows how the Flat Attribute affects the operation of Times in the simple example. Mathematica knows it can split up a function applied to two arguments times each other, and, typically for the evaluator, keeps applying the rule to split up all arguments of Times until there are no lists of more than two left.

In[104]:= log[2x y]//Trace

Out[104]= {log[2 x y],log[2]+log[x y],{log[x y],log[x]+log[y]},log[2]+(log[x]+log[y]),log[2]+log[x]+log[y]}

How to Write a Program in Mathematica: Basic Programming Principles

If you look at the code of the creators of Mathematica's programming language, Stephen Wolfram (e.g. Ch. 5., Cellular Automata, of A New Kind of Science) and Roman Maeder (see his superb books), you will see simplicity and organization. They are master programmers. So if the method of Wolfram and Maeder is to keep things simple and thoughtfully organize their programs, let us follow their example. With that in mind, here are some principles to create any program, step by step.

1. Write and debug series of one liners to execute your program. In other words, start by just doing what you want to do "by hand," one line at a time. Then when you're done and it works, assemble the series of one liners into a program.
2. If the input or output of any step is at all complicated, first solve a toy version. If your entire program is complicated, first solve a toy version of it. Then try each step on the actual problem.
3. Functional programming emphasizes writing each function with as few inputs and outputs as possible–ideally just one output. If you have only one input and one output per function, each is easy to write and debug. Sometimes you have to carry related parameters together. In the function under construction, it's still advisable to have as few parameters affect a single output as possible. 
4. To assemble the program, first create the shell, i.e. just a Module (or With or Block) with the name of the program function and those of its arguments. Then put each one-liner in the program one at a time, and run the program, returning or printing the expected output of that one liner. A Print statement is the simplest form of debugging. If you have a long set of arguments or if any are complex, start by just returning the arguments to see if you get what you expected. Leave a Print statement in the program if you need to monitor each output, or if not, delete it. If you think you might need it, comment it out.
5. And in general, per the 2000-year old army saying, Clean As You Go (which derives from the law of increasing entropy). Delete things you don't need. Re-organize as needed. 
6. There is no shame in clarity. Someone else may need to read your code. And you may be that someone else. As an acquaintance of mine, Gary Drescher, once said, "The 'I' of the future is only loosely coupled to the 'I' of the present." You, 6 months from now, will not have an easy time of deciphering your own code unless you go for clarity. 

• Use long variable names (like Hoare and Trott). 
• Format a cell for text (Alt+7) above your program and comment freely. 
• Use inline comments (* comment here *) judiciously alongside lines of your code. Too many comments break up the readability.
• Use the combination of Prefix and Postfix with pure Function that I advocate (here in this blog), which simplifies your code and gives you total control over which functions to emphasize and which to subordinate. 

7. As you put each one liner in the program, be sure to include any variable names in the local variable List in the Module. If variable names are not listed Mathematica will flag them in a color (blue in my version). You can save space and simplify things by putting assignment functions in the local variable List as long as they're not dependent on a variable that came earlier in the List. If they are dependent, the assignment has to be put in the program body. Put the local variable names in the List in the order that they appear in your program.

8. Remember the fundamental data structure principles:

• Write and debug a special function, called a Selector, to extract each subset of what you need from your data. Only use Selectors to touch the data; never touch it directly with the program. When the structure of your data changes, re-write the Selector.
• Write and debug a special function, called a Constructor, to build each construct from the data. Only use Constructors to build the constructs. Leave the original data alone.
• If need be, write and debug a special function to export or format the output of your program. Only use the special function to export or format your output.
• Two other arch-typical data structure functions are constants, that always return the same thing either with no input or no matter what the input, and predicates, that take an input and return True or False.

9. Whether to define a function in your program, or outside of it and then call it in the program, is a judgment call. A simple but valid answer is if you plan on using the function outside of your program, define it outside to facilitate that usage. But an equally important reason for defining a function outside of your program is to simplify it.
10. With a medium to large program, consider just throwing out the original version if you see a better approach. At an IEEE seminar on object-oriented programming, the speaker said often you just need to start coding for a bit to get a feel for what you want to do before you can envision a better architectural solution.

Obviously I need to provide examples to illustrate all these principles and will do so.

Introduction to Strings in Mathematica

In programming languages, "String" is the name for the data type of text expressions. In other words, unlike symbols in a general expression, the symbols in a String, like a, b, c, do not refer to places in memory that store a variable, like a function or value. Symbols in a String do not refer to anything, or you could say they only refer to themselves as "literals", that is, the literal characters or tokens "a", "b", "c". An "a" is just a blank token with nothing behind it; hence the terms "literal" or "token" for a String symbol.

In[21]:= a = 5534.7; a

Out[21]= 5534.7

Trying to use a String as a variable gives an error message.

In[22]:= "a" = "Lookee here."

During evaluation of In[22]:= Set::setraw: Cannot assign to raw object a. >>

Out[22]= "Lookee here."

Consequently, Strings are not evaluated in the usual way that other Expressions are. Wellin et al. in their excellent book suggest that we think of a String as being broken up by Mathematica into a sequence of its characters and then functions are applied to the sequence.

In[8]:= aSentence = "The quick brown fox jumped over the 742 lazy white dogs.";

In[11]:= aSentenceCharacters = aSentence // Characters

Out[11]= {T,h,e, ,q,u,i,c,k, ,b,r,o,w,n, ,f,o,x, ,j,u,m,p,e,d, ,o,v,e,r, ,t,h,e, ,7,4,2, ,l,a,z,y, ,w,h,i,t,e, ,d,o,g,s,.}

Functions, like MatchQ, Cases, Position, etc., that work on non-String expressions do not work on Strings but do work on their sequence of characters.

In[14]:= Position[aSentence, "e"]

Out[14]= {}

So perhaps as Wellin et al suggest, Mathematica first breaks a String into a sequence of characters and applies some of the regular built-in functions as well as some specialized String functions to the sequence.


In[25]:= MatchQ[#, "e"] & /@ aSentenceCharacters

Out[25]= {False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, True, False, False, False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, False, False}

In[24]:= StringMatchQ[aSentenceCharacters, "e"]

Out[24]= {False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, True, False, False, False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, False, False}


In[15]:= Position[aSentenceCharacters, "e"]

Out[15]= {{3}, {25}, {30}, {35}, {50}}

In[16]:= StringPosition[aSentence, "e"]

Out[16]= {{3, 3}, {25, 25}, {30, 30}, {35, 35}, {50, 50}}

The position index is repeated in StringPosition since it shows the beginning and ending positions of the matched String.

In[20]:= StringPosition[aSentence, "lazy"]

Out[20]= {{41, 44}}

When working with Strings, you do not need to enclose a String in List brackets:

In[3]:= aString = {"abcd"}

Out[3]= {abcd}

In[4]:= aString2 = "abcd"

Out[4]= abcd

In[5]:= Head@aString2

Out[5]= String

The String is delimited by invisible quotation marks. As usual FullForm and InputForm reveal the way Mathematica sees the String.

In[6]:= FullForm@aString2

Out[6]="abcd"

In[7]:= InputForm@aString2

Out[7]="abcd"

Next we will look at the variety of String functions in Mathematica.

How to Lose List Brackets in Mathematica

Apply, Sequence, ReplaceAll

There come times in every Mathematica user's life when the all-pervasive List brackets get in the way. Sometimes it's easy to redesign the function to accept the arguments in a List. Here are three other solutions.

Apply

First, see if you can use Apply @@ to apply a function to the contents of your List. Apply will replace the List head with your desired function and Evaluate it.

In[107]:=  Range@15
Out[107]=  {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}

In[108]:=  Plus@@%
Out[108]=  120

Here I have found a single filename that is unnecessarily surrounded by List brackets. It's a String, so I just Apply ToString to it. It then stands alone--no List brackets--but it's Head is String.

FileNames[]

{TableFiredFibers-Control-CathodeOnly-0p644.txt}

ToString@@%141

TableFiredFibers-Control-CathodeOnly-0p644.txt

Head@%
String

FullForm@%%
"TableFiredFibers-Control-CathodeOnly-0p644.txt"

Sequence

Mathematica was designed as a list processing language in which the list is the fundamental data structure. Allan Newell's IPL (Information Processing Language) was the original prototype, after which John McCarthy coupled the list as a universal data structure with the lambda calculus and logic functions into the major programming language, LISP. Let me note in passing that Sequence is the more fundamental data structure in Mathematica, being the container of all arguments including those within List.

You can Apply Sequence to a List to replace List as the container (the Head). Here are two examples, first to a single List, then to a List of Lists.

In[115]:= Sequence@@{0,4}

Out[115]= Sequence[0,4]

In[119]:= Sequence@@{{0,1,1},{0,2},{0,1,3},{0,4}}

Out[119]= Sequence[{0,1,1},{0,2},{0,1,3},{0,4}]

FileNameJoin is the safe way to assemble filepaths in Mathematica since it knows your operating system syntax. However FileNameJoin only accepts a single-level List as its argument. If you need to feed it a nested List, however, you can use Sequence instead of List as the Head of the inner List.

In[350]:= twoPathElements={"webPages","index.html"};

In[351]:= FileNameJoin@{$HomeDirectory,twoPathElements}

Out[351]= FileNameJoin[{C:\Users\kwcarlso,{webPages,index.html}}]

Since it didn't recognize the correct argument types, FileNameJoin returned my input. Here it is using Sequence.

In[352]:= twoPathElements=Sequence["webPages","index.html"];

In[353]:= FileNameJoin@{$HomeDirectory,twoPathElements}

Out[353]= C:\Users\kwcarlso\webPages\index.html

The context of one function I use to illustrate Sequence is worth mentioning (but skip this to get to more usages of Sequence). I have digitally sampled a curve from the literature (a paper by Holsheimer from 1999) in the graphing program Origin. In other words, you can import an image, in this case a graph for which Holsheimer doesn't give either the underlying data or an equation, and repeatedly click on it to harvest {x,y} values. We captured 163 sample points on the solid curve below, which is a vast improvement on the antiquated method of using a straightedge to pick up x and y values off the axes.

With that many sample points, finding the x value closest to the one desired is a reasonable alternative to fitting a curve and applying the fit equation to values you wish to interpolate or extrapolate.

So this is a function to find the nearest x value (a neuron axon diameter, fiberDiameter) to one for which we want to know the threshold as determined by Holsheimer (which is the y-value retrieved). The problem is Nearest returns values in List brackets, which don't match the un-bracketed x values in the List of data. Applying Sequence to the result of Nearest feeds the unbracketed value to Cases, which then performs the match.

Clear@diameterThreshold; diameterThreshold[fiberDiameter_, fiberThresholdSourceData_:holsheimerData1999] := Cases[fiberThresholdSourceData, {x_, y_} /; x == Sequence@@Nearest[fiberThresholdSourceData[[All, 1]], fiberDiameter] ]

So in sum this function says: Clear the function name (which I do habitually at the start of a function definition to prevent accidentally testing a previous version as I revise it). Then define it as the Cases in the source data file (fiberThresholdSourceData--default to the holsheimerData1999 file unless I specify another file) such that ( /; ) the x-value is Nearest to the fiberDiameter parameter I give it.

ReplaceAll

Here is a third way using rule patterns.

In[128]:= {{0,1,1},{0,2},{0,1,3},{0,4}}/. List[a__] ->a

Out[128]= Sequence[{0,1,1},{0,2},{0,1,3},{0,4}]

Mathematica Keyboard Shortcuts

Keyboard Shortcuts and Handy Short Commands

Here are the shortcuts that I find most helpful. These are a combination of the most useful and least known. Note that I do not recommend the cumbersome and difficult-to-read functional programming syntax currently used by all Mathematica users, for instance featuring brackets for single-argument functions. This style is a waste of typing and more difficult to read than using the Prefix-style I champion: function@argument.

Likewise, I have little sympathy for the prevalent use of increasingly distal matched brackets to indicate deeply nested functions, which is also difficult to read, increases programming time, and doesn't allow the programmer to highlight what he or she feels is important. Since my 2007 User Conference talk (also here) I have advocated Prefix with single arguments and Postfix-style with pure Function for nested functions:

yFilesData[[1]] // Rest // Flatten // Length;

yFilesDataDataDifferences//Chop[#, 10^-4]&//Union/@#&//ListPlot/@#&//Partition[#, 3]&//Grid

Functional-Procedural Fusion" (I am open to a better name) elegantly represents deeply nested functions in a readable manner. But if you do use deeply nested functions, sensei Paul Abbott showed me that multiple-clicking within the Expressions will select matching brackets outward.

Ctrl+F6          Switch between open Mathematica notebooks
F12                Toggle current notebook to and from full screen

Ctrl+Shift+K   Gives you a template for the simplest form of the function to its left

Ctrl+K            Gives a drop-down listbox with autocompletions of the function name you are typing

?*name*         Lists all Mathematica functions using * as multiple-symbol wildcards. You can then click on them to get quick help text.

?@                 Lists all names created in current Mathematica session

F1                  Search Help for selection or expression to left of cursor

Shift + F1       Open new instance of Help Browser leaving current instances alone

Alt+.               Interrupt a calculation. If it doesn't work, do Evaluation/Quit Kernel. You can then Evaluate what you need from your current session and if you wish, clean it up by deleting cells. However, I advocate the 2000-year-old army policy, clean as you go.

%+Out line    As: %127. Refers to output line #127. Same as Out@127 or Out[127] but more concise. I use this frequently for a temporary variable name but if it turns out I need a more permanent mnemonic name I just create one: newName = %127. I use %, %%, or %2 only for very temporary "scratchpad" calcs.

Clear@Out@127  Removes from memory some huge expression you created or imported in In[127].
ClearAll["Global`*"] and Remove["Global`*"] Clears or Removes all values from variables you created in the current session. If it comes to that, I usually kill the kernel though (Evaluation/Quit Kernel).

Ctrl+L            Copy input cell from the one just above
Ctrl+Shift+L   Copy output cell from the one just above, into new input cell. An alternative is to just start typing in the Output cell and Mathematica automatically creates a new Input cell with those contents.
Shift+Ctrl+P   Print current selection

esc+[character}+esc  Usually gives you the Greek letter corresponding to the keyboard character

filePath="C:\\directory1\\directory2"; FileNameSetter@Dynamic@filePath

Gives you a in-Notebook button to click to navigate and select a filename or directory to which to Set filePath. Giving filePath an initial value before evaluating FileNameSetter takes you to that value (e.g. a directory) when you click the button. Using Dynamic lets you click the button over and over to re-Set filePath.

CreateDocument@expression Opens a new Mathematica notebook with expression in it. Expression can be an imported document.

Mathematica Plot: Using Table to See Numerical Values

Plot with Table to See Numerical Values

We saw how to use Table to generate sample points for a ListPlot, and how to compare your own sample to Plot showing its mesh points. Here is the converse usage, using Plot to graph a function and then Table to see it numerically. We'll start with a simple sine curve.

To see numerical values, I click Control+L to regenerate the input, replace "Plot" with "Table", and specify a third argument for the iterator to give the sampling frequency. The Postfix function N tells Mathematica to give us decimal output instead of its default, exact fractional values.

In[314]:= Table[{x, Sin@x // N}, {x, 0, 2 Pi, Pi/4}] // TableForm

Let's look at a more complicated example with two decay curves that are identical except for their asymptotes, which are 0 for decay1 and 0.7 for decay 2 (shown in the Plot).

In[347]:= Clear@decay1; decay1@d_ := (q/d) + 0.7; q = .2;

In[348]:= Clear@decay2; decay2@d_ := (q/d); q = .2;

In[363]:= Plot[{decay1@d, decay2@d}, {d, .1, 4}, PlotRange -> {Automatic, {0, 3}}]

When you need to keep track of three or more columns, TableHeadings can be helpful. Here we say no row headings, but three column headings.

Table[{d, decay1@d, decay2@d}, {d, .1, 1.2, .1}] // 

 TableForm[#, TableHeadings -> {None, {"d", "decay1@d", "decay2@d"}}] &