ARSC HPC Users' Newsletter Number 422 2013-06-19
The Arctic Region Supercomputing Center Users' Newsletter provides a
platform for discourse relevant to users of high performance computing
systems. Topics include: programming, commands, tools, applications,
Please send items, ideas, announcements, etc. to email@example.com.
To subscribe or manage your subscription, visit:
Welcome to the New Newsletter
[ By: Greg Newby, ARSC Director ]
Welcome to issue #422 of the ARSC HPC Users' Newsletter, which has
been on hiatus since the last issue was released on November 11, 2011.
A lot has happened at ARSC in two years. We have reduced our role as
a DoD contractor while increasing the depth and breadth of our
University of Alaska academic services, especially on our home campus,
the University of Alaska Fairbanks.
I'm pleased to bring you this new issue, and plan to maintain the
newsletter with articles of interest to ARSC users and the broader
community of high performance computing enthusiasts.
-- Greg Newby, ARSC Director
Getting into and out of Trouble with Git
[ By: Kate Hedström, ARSC Oceanographic Specialist ]
This article describes some of my experiences with git, a distributed
version control system. The git command line client is available on
ARSC systems. Here are the steps that got me into trouble:
1. Create private branch, check in private stuff
2. Apply a few fixes and check them in
3. Create new directory for new case
4. Decide the new stuff should be on the master branch
5. Check out master branch
So far, so good. The new directory is still there, not tracked by
git. But those few fixes should be in the master branch too. Let's try:
6. git checkout private
7. git log (get the sha1 number of last check-in)
8. git checkout master
Now we tell git about the new directory:
9. git add Apps/Arctic
10. git merge <sha1 from fixes>
Oops! A git status shows that many things changed - all those private
updates came along for the ride (and introduced a few conflicts). I
wanted a cherry-pick, not a merge. No big deal:
11. git reset --hard
12. git cherry-pick <sha1 from fixes>
The "git reset --hard" would have been no problem if I had never done
the "git add Apps/Arctic". Without the latter, git was not tracking
that directory at all, simply leaving it alone. Because I had entered
it into the Index, then decided to fall back to HEAD, it was tossed.
We got rid of that whole bad merge and got our cherry-pick. But wait,
our new directory has vanished without a trace! It's not in the master
branch, it's not in the private branch, and it's not in the index.
So what to do? I could try to reproduce the lost files, but I know git
must have them somewhere because they were temporarily in the index. A
web search on things like 'git recover' lead me to this great web
He talks about a similar but different problem, how to recover lost
commits, but the tip of running 'git fsck' is a lifesaver (I got the
same from simply 'git fsck' as from 'git fsck --lost-found'). It
listed over 40 'dangling blob' lines and one 'dangling tree'. I ran
'git show' on all of them and managed to extract five of my missing
files. I had a few more, but they must have been exact duplicates of
other files in my repository.
13. git fsck
14. git show <sha1 from a dangling blob>
15. git show <sha1 from a dangling blob> > Apps/Arctic/filename
Note that I had to provide the filename. The blobs are attached to
filenames in the tree object - the one dangling tree I had was not the
one I needed.
Once I had recovered all my files, I cleaned up my repository with:
16. git gc
I then tried 'git fsck' again and I was down to only eight dangling
blobs, including the five I needed. This makes the search go faster,
but I don't know that I'd have the guts to start with 'git gc' the
next time this happens to me.
Conclusions: Check in early, check in often, but all is not lost if
you mess up.
Practical Emacs Part I
[ By Christopher Howard, ARSC Research Project Assistant ]
Emacs is available on all user-accessible ARSC systems.
This article assumes the reader's familiarity with basic Emacs
navigation, notation, and terminology. If the reader is not familiar
with these concepts, please start Emacs and type Control-h followed by
t to launch the Emacs tutorial. Also, please refer to the Emacs
reference manual (C-h r) for a more detailed description of the topics
in this article.
First, determine which version of Emacs is available with the shell
command "emacs --version". Versions 21.* and earlier lack many of the
described features and sometimes use different key combinations.
Version 23.* or greater is recommended. The author uses 24.3.1. If
the system does not have a recent version of Emacs, it is not
difficult to build and install one from source; however, that process
will not be covered in this article.
Macros are without doubt the most versatile and practical feature
provided by Emacs, and they are sure to be used frequently. A macro is
basically a set of key presses or commands that can be created easily
and then repeated quickly.
To create a macro, press the F3 function key. Enter any combination of
characters and Emacs commands. When done defining the macro in this
way, press F4 to finish recording it. After this, each time the F4 key
is pressed, the last macro defined will be repeated exactly as
An aside: Some terminal applications and operating systems do not allow
Emacs to use these function keys. If that is the case on a system,
Emacs provides an additional set of key bindings for macros:
C-x ( Begin recording C-x ) Finish recording C-x e Use last recorded macro e Repeat macro (after first use)
With a little practice, this "on-the-fly" use of macros will allow one
to quickly perform some otherwise complicated text transformations and
to avoid contriving complicated regular expressions or substitution
commands. For example, imagine the following lines of text:
www.example.com; Example Site www.example2.com; Example Site 2 www.example3.com; Example Site 3 ...
Positioning the cursor at the beginning of the first line, the
F3, "<a href='", C-s, ";", ENTER, DEL, BACKSPACE, "'>", C-e, "</a>", C-n, C-a, F4, F4, F4...
will allow the user to quickly transform the above text into
<a href='www.example.com'>Example Site</a> <a href='www.example2.com'>Example Site 2</a> <a href='www.example3.com'>Example Site 3</a> ...
It is also possible to apply macros regionally and to save macros for
future use. See the Emacs manual for more details. (C-h r m Keyboard
Normal search-and-replace in Emacs is done with the command M-x
replace-string. This is very handy, of course. Yet, sometimes one is
not sure whether it is safe to replace every instance in the buffer.
In such cases, use M-x query-replace.
The query-replace command operates in the same manner as
replace-string, except that the user is given the opportunity to
decide what will be done with each replacement opportunity. Press y to
replace and n to skip a replacement. Press ENTER to stop making
replacements. Press . (period) to make a final replacement and then
Quick Shell Commands
Sometimes, a user need to execute a simple shell command, but it is
inconvenient to open another terminal. For convenience, run the
command M-!, followed by the text of the shell command. The output of
the command will be displayed in the echo area.
For more complicated interactions, open the system shell inside Emacs,
with the command M-x shell. History can be accessed with M-p
See the Emacs manual for many more details. (C-h r m Shell)
Please note: because opening a shell buffer will take control of the
active window, the user may first wish to use the command C-x 2. This
will divide the Emacs window into two windows so the shell buffer does
not capture the entire display.
Imagine receiving someone else's source code file and discovering the
indentation is very messy. Simply run C-x h to select the entire
buffer and then C-M-\ to bring the text into conformance with the
proper indentation for the active Emacs mode.
This command, in combination with the command M-x
delete-trailing-whitespace, can be helpful for cleaning up source
The command M-x compile will launch any compile command desired in the
current working directory. (The default is "make -k".) The advantage
to compiling this way is that if the compile fails, the errors will be
displayed in a separate buffer window in Emacs. If the compiler
outputs the usual error prefix (file name, line number, column number)
the user can move to each error location in the source code by
pressing C-x ` (backtick) repeatedly.
Enter C-h r m Building m Compilation for more details.
Quick-Tip Q & A
These were the Quick-tip items from issue #421. Since responses
were never published, let's see whether there are some new ideas.
Q: I have some Fortran 90 code I want to call from inside Python. I have a setup.py file that can be used to build it, but I need to invoke it with "python setup.py build --fcompiler=gnu95". Is there any way to put the fcompiler option into the setup.py file, at least as a default?
No answers were forthcoming. We put the need for --fcompiler=gnu95 into the README file.
Q: I need to copy some large files between computers. What is the fastest way to do that?
[[ Answers, Questions, and Tips Graciously Accepted ]]
The Arctic Region Supercomputing Center (ARSC) provides research and
high performance computing, large-scale storage, and related services
to the students, faculty and staff of the University of Alaska.
Greg Newby, ARSC Director, firstname.lastname@example.org, 907-450-8663
Monthly, on approximately the third Wednesday.
Subscribing and unsubscribing:
Quick tip answers and other correspondence: