FAQ

 

NCF Questions

  Getting Started

  Basic UNIX Topics

  Advanced Topics

  SLURM Topis

  NCF Troubleshooting

  Old LSF help topics

CBSCentral Questions

Quality Control

  Overview

  Quantitative QC: Extended BOLD QC via CBSCentral

  Qualitative QC: Looking at your data

  Preventive QC: at the scanner

Matlab Questions

Scanner Questions

        Getting Started

      Interactions with your Experiment

       SMS/Multiband topics

      Operating the Scanner

        Scanner Troubleshooting

Basic MRI Physics and Protocol Questions

How can I get help?

For assistance with NCF related issues contact: rchelp [at] fas [dot] harvard [dot] edu or call (617) 299-9724 and leave a message. Please provide as much detail as possible so that the Helpdesk staff can address your problem quickly. In addition, Nate St. Germain from RC has office hours on Tuesday and Thursday, 1:00-3:00 PM in the mini cluster on the second floor of the Northwest building, east wing, by the kitchen.

Questions about network, desktop or laptop support should be directed to Harvard University IT support (HUIT) at ithelp [at] harvard [dot] edu or (617) 495-7777.

For questions about CBSCentral (XNAT) or the NRG tools (i.e. fcfast) contact the neuroinfomatics group at at cbssupport [at] neuroinfo [dot] org.

For questions about the scanning facility, including safety issues, contact Tammy at tmoran [at] fas [dot] harvard [dot] edu, and see here.

For questions about setting up scan parameters or MR physics questions, talk to Ross at rmair [at] fas [dot] harvard [dot] edu.

For questions about experimental paradigm design, data analysis issues, or comments/questions about anything on the FAQ page, contact Stephanie at smcmains [at] fas [dot] harvard [dot] edu.

What is the NCF?

The Neuroimaging Compute Facility (NCF) is a central enabling infrastructure for neuroimaging teaching and research whose mission is to provide high performance, high power, robust, reliable and secure computer systems and human expertise to meet the challenges of neuroimaging research and teaching. The NCF is a collaboration between the Center for Brain Science and the Life Sciences Division of Research Computing. It consists of a collection of workstations and a compute cluster.

Who may use the NCF?

The NCF is a resource for the Harvard Neuroimaging Community and their collaborators. To access, just sign up for a user account indicating your laboratory and Principal Investigator.

How do I use the NCF?

There are several main uses for the NCF. With modern technology and advanced analysis techniques, datasets can be very large and can often fill up a users’ personal computer quickly. Therefore, the NCF provides a safe and secure location to store data. In addition, the data is backed up automatically and regularly, providing peace of mind. Second, the NCF has several analysis packages that are maintained to help users analyze data. This is done in two main ways. First, the compute cluster can be used to run computationally intensive jobs via the command line that do not require user input or interactions with a GUI. Second, the workstations can be used for data analysis that requires interaction from the user or the use of a GUI interface. The fastest way to run programs like this, such as SPM, is to sit in front of a workstation. In addition, users can login remotely from their own machine via VPN and x11 forwarding. However, this can be slow depending on the program being used. Another possibility that some users have reported is faster than logging in remotely, is to set up a VNC session, a type of virtual desktop program, that allows you to ‘take over’ a workstation from your desktop. Importantly, this doesn’t prevent someone else from using the workstation.

What should I acknowledge or cite when writing up my work?

ALL published work resulting from research studies fully or partially undertaken at the Center for Brain Science must include formal acknowledgement, using the standardized language below, of support from the Harvard Center for Brain Science. If the research involves data collected on the Prisma MRI scanner, the NIH Shared Instrumentation Grant must also be acknowledged.
Acknowledgements should also be included on any posters or presentations.

Please use the following language when acknowledging the Center and the NIH instrumentation grant in all publications.

“This research was carried out in whole or in part at the Harvard Center for Brain Science. This work involved the use of instrumentation supported by the NIH Shared Instrumentation Grant Program; specifically, grant number S10OD020039.”

If you used the SMS/multiband sequences, you should also cite them. Instructions for this can be found here.

How do I get an NCF User Account?

To request a new account, fill out the online Account Request form. If you select a PI that is for the NCF, the last page should say that NCF compute resource access has automatically been included. If for some reason it doesn't, please email RC (rchelp [at] fas [dot] harvard [dot] edu) and let them know.

If you are an outside user trying to just get access to the Spinal calendars, select Users without a Harvard Key, Caroline West as your PI, and then select the Spinal calendars on the instrument tab.

You can also go to the second tab on that page, instruments, and request access to the Spinal calendars.

How do I set up my user environment/access software on the NCF?

If you are working in bash, which is the default on the NCF, you should have a file in your user directory called .bashrc. You can open/create this file using your favorite text editor, such as gedit. Copy and paste the lines from below.  This will get you all the default programs, including the NRG tools (fcfast, procfast), and many brain analysis packages.  Many of the defaults are very old versions, given that people often don't want these to be randomly changing on them.  To see how to select a different version, go here.

# writen by Luis Silva 02/10/2012
# For assistance please contact rchelp [at] fas [dot] harvard [dot] edu
#
#################################
# NCF Base setup install script #
#################################
#

#default environment, by using 'current' it will be updated automatcially with new releases.  This should not change the default versions of software without assuring backwards compatibility.

. /ncf/tools/current/code/bin/env_setup.sh

#this makes your history file larger and easier to read so RC can more easily help you
HISTFILESIZE=10000
HISTCONTROL=ignoreboth
HISTSIZE=10000

#This makes your command prompt pretty, including having the pathname to where you are

PS1='\[\033[0;36m\]\u\[\033[0;35m\]@\h:\[\033[0;36m\]\w\[\033[0;35m\]\n[\!]\[\033[0m\]';

#
##################################################################
# Insert additional configuration below this line, like module loading and alis setting #
##################################################################

 

If you have questions about setting up your environment, you can contact Stephanie (smcmains [at] fas [dot] harvard [dot] edu) or RC help (rchelp [at] fas [dot] harvard [dot] edu). See below for how to change versions of software via environment variables, or modules.

 

How do I access the NCF network remotely (VPN)?

The latest Neuroimaging Compute Facility (NCF 2.0) uses two-factor authentication, which requires users to use their username and password as well as second factor, which is now (5/18/2012) an openauth token, in order to gain entry to NCF infrastructure. This is not the same two-factor authentication you have to do for the Harvard key system. Its also not the same vpn server, though you can use the same vpn app, just with a different address.

You must first install the openauth software.  The link below will provide instructions for how to install this on your smart phone and/or your computer. This software creates a token that is specific to you, so if you use a different computer, you will need to set it up again, or use the token from your phone.  Therefore, if you plan on VPNing to the cluster from multiple locations it is highly recommended to install the phone app also.  Just follow the instructions on the page below, until you get the java application installed, then continue on with these instructions to see how to VPN. Importantly, they provide two options for download, most likely you want the second one, which provides a more traditional application for you to download, with an icon for your desktop. The other one creates more of a file (JAuth.jar) that you need to run.

https://software.rc.fas.harvard.edu/openauth 

Once you download the software click on the JAuth.jar (or JAuth.app) file to launch the program.  This will make a little black and green box appear with a 6 digit code that is updated every minute or so. This code will be needed to successfully VPN to the NCF.

If you have a new Mac, you might get an error message telling you to download the Java Developer Kit. This means you will need to install java to get the program to run. This can be downloaded from here.

Now that you have the openauth token generator, you will need the actual VPN client. These steps will only need to be followed once per machine you want to install it on, and this application is not specific to a user, so if you find it already installed on a machine you can use it if you have your own personal openauth token. It is also the same one you use to VPN more generally to Harvard, you just need to change the address as listed below. To get the VPN client (if you don't already have it) go here. Again, you will only have to do this step once, as this will install the Cisco AnyConnect VPN Client onto your system.

The web based installation of the client almost always fails. However, after it fails, it gives you the option to download the disk-image. Go ahead and do that. When you try and open the installer package, it may give you an error that it is broken and can't run. This is usually due to your security preferences in your web browser.

On a mac:

  • Go to System Preferences -> Security & Privacy
  • Click on the General tab
  • Click on the lock to allow changes
  • Under the heading "Allow applications downloaded from:" chose Anywhere

Then double lick on the vpn.pkg and the installation should work. Afterwards, you can go back and set the "Allow applications downloaded from:" to be Mac App Store and identified developers.

Once you launch the VPN client, make sure the "Connect to" path name at the top is vpn.rc.fas.harvard.edu

If this isn't in the drop down menu, you will have to type it in. It should then be remembered for future sessions.

Next you will be prompted for your username: @ncf i.e. jsmith@ncf, DON'T FORGET THE @NCF!! Next you will be prompted for your password: The second password field is for the 6 digit passcode from your openauth app. Once you enter this information, you should be able to connect.

Occasionally, you may have to try logging in once or twice, particularly if the token is getting ready to expire right as you put it in. Once you are connected, you can use a SSH client, such as putty on a PC, or terminal on a MAC.

Open an SSH session:

ssh –YC username [at] ncfws12 [dot] rc [dot] fas [dot] harvard [dot] edu There are several workstations (other than ncfws12) you can log into, with a list and instructions here.

Why do I have to use an NCF designated workstation and where can I find one?

The NCF system is a secure private network. Dedicated high-performance workstations are set up and maintained by NCF staff for data management and image analysis. You may access the system via one of the designated terminals located in the MRI suite area in Northwest B4, in the Northwest office suite 280-295 area, and at the Center for Brain Science Neuroimaging pod on the second floor of William James Hall. See here for a list of designated workstations.

How and where do I login remotely?

In order to login in remotely, you first need to have your VPN access setup and going. Before logging into a workstation, it is advisable to check the usage on the machines in order to pick a machine that is not being heavily used. This can be done via a website (you MUST be on the VPN). Scroll down until you see the little graphs of the individual work stations.  The work stations all begin with ncfws.

If you want to use the old LSF cluster, your only choice is: ncfws01.rc.fas.harvard.edu.

 The newest workstations are:

Hostname: ncfws12.rc.fas.harvard.edu

 Hostname: ncfws13.rc.fas.harvard.edu

Hostname: ncfws14.rc.fas.harvard.edu

Hostname: ncfws15.rc.fas.harvard.edu

 

However, several older machines exist also:

ncfws04.rc.fas.harvard.edu

ncfws05.rc.fas.harvard.edu

ncfws06.rc.fas.harvard.edu

ncfws09.rc.fas.harvard.edu

ncfws16.rc.fas.harvard.edu

ncfws20.rc.fas.harvard.edu

Once you decide on a machine, you need to SSH.  On a Mac, this can be done via the Terminal, which is located in the Applications folder, inside of Utilities.  For a PC, see directions below

At the Terminal prompt, type:

ssh –YC ncfusername@ncfws??.rc.fas.harvard.edu

Where the ?? would be replaced by the workstation number. 

 

How do I connect to the NCF from a PC?

On a PC you will have to download an X11 clienet, and a SSH client.  The preferred X11 client is Xming, downloaded here.  Next, you have to download the SSH client, PuTTY from here. To enable X11 forwarding in PuTTY you need to configure your settings.  

Start up PuTTY.  In the configuration box you need to give it the Host Name of a ncf workstation.  A list of available numbers (to replace the ??) can be found here.  You can also save these settings by giving your session a name in the Saved Sessions box and then click Save.  You should save it again once you set the X11 forwarding.

Next, under the Connection Category, expand SSH (click where red arrow pointing) and choose X11.  Click on the Enable X11 Forwarding checkbox (where red arrow pointing).  

Then, click on the Session Category, and save your settings again.  Next time when you open this program, you can Load the saved settings.

Finally, click on Open.  You should then be prompted for your username and password.  When you are done, make sure you end your SSH session by typing exit at the command prompt before you X out of the PuTTY program.


 

How do I run GUI intensive programs from my laptop - VNC and NX?

There are two options for this, VNC and NX (being phased out). Some graphical programs will run faster on one or the other, so if something is slow, try the other program. If you find something really needs NX to run, please let us know. The biggest thing you can to speed things up is to GET A WIRED CONNECTION!!! The wireless network is much slower, and while convenient, is not great for remotely working with the cluster. We also now have the ability to take advantage of virtual GL which speeds up graphic intensive programs. Make sure you see step 6 below for how to take advantage of this.

Also, please keep in mind VNC should be more for graphic intensive processes. If you need to do number crunching, make sure you are using the compute cluster, either by submitting your scripts, or if you still need graphics (like with matlab), you can run an interactive sessionfrom your VNC session.

VNC

1. You need to install a VNC program on your laptop, such as Tiger VNC, Real VNC, or Chicken of the VNC, which is used in the example below.

To take the most advantage of the new virtualgl graphics acceleration, you should use Tiger VNC. It will direct you to a github page, which currently takes you to a bintray site. Scroll down the list until you see a .dmg (Mac), or .exe (PC) for installation.

2. [CHANGED] Connect to the VPN and log in to the vnc server

ssh –Y username [at] ncfvdi [dot] rc [dot] fas [dot] harvard [dot] edu

3. Set up a VNC server:

vncserver

This will prompt you for a password, we recommend you use your NCF, but you can use what ever you want, just remember it because you need to enter it into the VNC software.

4. Get your display number:

This command above will return your display number in an output that looks like:
New 'ncfvdi.rc.fas.harvard.edu:2 (username)' desktop is ncfvdi.rc.fas.harvard.edu:2
The last number is your display number (2)

4. Launch your favorite VNC program. The instructions here use Chicken of the VNC. Importantly, a new time limit of 48 hours has been implemented , see below for more info.

Enter the required information:
The host name (e.g. ncfvdi.rc.fas.harvard.edu).
The display number reported by vncserver (e.g. 1)
Enter your password
Choose “Full Screen” if you want it.

NOTE: For TigerVNC and RealVNC, launch the 'viewer' application and the display number goes after the host name, preceded by a colon. (e.g. ncfvdi.rc.fas.harvard.edu:1)

 

If you choose full screen, you can exit it by typing cntrl+command+alt+’

5. End your VNC session. There is now a 48 time limit on your VNC session as we have one server instead of several workstations hosting these. That means the best thing to do is treat it like your session will end when ever you exit out of your local VNC application. If you want to see how long a session has been active, you can check it from ncfvdi:

vncserver -list

You will get output that looks something like this:

owner => display => process-ID => elapsed-time
mcmains | :2 | 1654 | 01:04:48

Sometimes you might see an old session that still lists a display and process ID but no elapsed time. This means the session has been killed. Please start a new one.

If you want to kill it yourself for some reason you can:
vncserver –kill :<display_num>

vncserver –kill :1

6. To speed up your graphics:

Insert the command vglrun before you launch your graphics intensive program (like matlab, fslview, freesurfer). Please keep in mind, this is not for crunching numbers, but just for looking at data or minimal number crunching. Most memory or time intensive programs should be run on the cluster. If you need graphics, you can run via an interactive session.

vglrun matlab &

6. Troubleshooting! Most issues can be solved by deleting your local VNC directory with your preferences. This includes deleting your password. So you can do this if you forget your password. It will just prompt you next time you launch a VNC session from ncfvdi and you can choose a new one. This also includes if copying and pasting between your local machine, and if the session doesn't look right when it opens up.

From your user directory type:

rm .vnc/*

NX

NX is eventually going to be phased out along with the workstations, so it is best to switch to the VNC server described above. However, if you have issues running something via VNC it is something to try. But you should let us know so we can try and fix issues, as again this will eventually go away as an option.

To use NX, which is a desktop sharing program, you need to install the noMachine client. This can be downloaded from here. Once you have noMachine installed, follow the instructions below.

1. Connect to the VPN.

2. Start noMachine. When it first starts up, it is an introduction screen, you can click continue.

3. Setup a new connection by clicking on the Add Computer button.

4. For the protocol select SSH, click continue.

5. Enter the Host as:
ncfnx.rc.fas.harvard.edu
and the Port as:
22
Click continue.

6. Select Use the NoMachine login, click continue.

7. Leave blank and click continue.

8. Select Don't use a proxy, click continue.

9. Choose a name that makes sense to you. You can create a link on your desktop if you want. Click Done.

10. It will bring you back to the original screen, but now you should see the computer configuration you just made and named. Select it and click Connect.

11. Enter your NCF username and password. You can save your password if this is your own computer. Click OK.

12. Select Create a new GNOME virtual desktop. You can also check Save this setting in the connection file. Click Continue.

You will now get a series of helpful screens telling you how to interact with noMachine. The key one tells you how to get the menu options, so you can do things like make it full screen or not. Click through these (Reading them of course!), until you get your session. It will look like it does when you are sitting in front of a workstation. To get a terminal window, you can right click on the desktop/background as usual.

To get the noMachine menu, you can hover your mouse in the upper right corner:  

How to quit!
When you are done, you should close the window. This will pop up a message that asks you if you want to suspend, terminate, or cancel. If you suspend, anything you have open will stay open. Make sure you don't leave a bunch of sessions suspended. Normally you want to terminate, which would be the equivalent to logging out of the workstation. All of your windows and programs will close.

If you don't close the window and instead quit the application, your session will automatically be suspended.

When you start noMachine again (or if you leave applications open like me, select new window), you should see the computer/configuration you setup and named. Select it and click connect. You will then either start a new session, or be brought to a screen where your old suspended session is available for selection.

What software is available?

Common tools for data analysis are available and updated regularly. Licenses and software packages for data analysis include AFNI, Analyze, FreeSurfer, FIV, FSL, MatLab, and SPM. Neuroinformatics tools including XNAT and Oasis are also available. In addition, a developing shared resource will house common processing scripts and tutorials for neuroimaging data analysis including merged atlases for analysis of older adults and children, as well as innovative procedures for high resolution data analysis and surface visualization.

How much space will I be allocated to process and store my data?

Work and data storage space are allocated to the Principal Investigator of each laboratory. Within a lab, the PI is responsible for dividing up space, so see your PI for space requests. In addition, each user will have 20 GB of space in their home directory. Please do not use the home directory for data storage - it will run out fast!  And definitely don't run jobs from your home directory as it can slow everyone's account down.  In that case your account could be deactivated.

How do I set my permissions so that everything is readable and writable by my group?

Initially, your permissions on the cluster are set up so that new directories and files are readable and writable to you, and readable by your group, and by the world (in this case, everyone with access to our cluster). You can tell what the permissions of a file are by typing ls -l. You will see something like this:

For a file:

-rw-r--r-- 1 mcmains cnl 54 Oct 5 2012 file_name

For a directory:

drwxr-xr-x 1 mcmains cnl 54 Oct 5 2012 directory_name

For a script: -rwxr-xr-x 1 mcmains cnl 54 Oct 5 2012 script_name

The first slot is either - for a file or d for a directory. The next three slots designate the owner's permissions (r=readable, w=writable, x=executable). The next three are the permissions for the group, and the last three are everyone else. Following this is the user name of the owner (mcmains) and then the owner's group (cnl).

Often, you will share files with other people in your group. In this case, you might want other members of your group to be able to read and write (edit) your files. In addition, you may not want other's on the cluster to be able to read your files. The default permissions can be set by adding a line to your .bashrc file. What you add depends on the permissions you want.

The command is umask followed by three numbers that designate the permissions separately for the three designations (users, group, everyone else).

0=rwx   1=rw   3=r  7=nothing 

So if you wanted to make your files readable, writable, and executable by you and your group, but no one else, you would add the line:

umask 007

to your .bashrc file. For more information on permissions, including how to change them for just a single file or directory of files, see: https://en.wikipedia.org/wiki/Chmod

How do I get files back and forth between the NCF and my computer?

This can be done in two main ways. The first is via the command line via sftp.

 

1. Command line:

 

Start an sftp session from your laptop. To do this you will need to have a VPN session going, and a terminal window open:

sftp username@ncfws??.rc.fas.harvard.edu

Where the ?? is replaced by the number of the workstation you want to remotely login to. For choices, see here. You will be prompted for your password. It is important to note that it is easiest to start the sftp session from the directory that contains the files you want to put on the NCF or from the directory where you want files from the NCF to be placed, as the starting directory is the default directory and ‘tab completion’ doesn’t seem to work (at least not for me).

If you want to PUT files from your laptop to the NCF, go to the directory on the NCF where you want the files to end up and then type:

put <filename>

If you want to get files from the NCF, go to the directory where the files are (cd, ls, and most unix commands should work). Then:

get  <filename>

You can put or get multiple files using the * character to select multiple files.

 

2. GUI based:

 

You can also use a program to link the NCF to your computer so that you can surf through the directory structures easily on both, and just drag and drop files or folders that you want.  

There are a few free downloadable choices. Research computing has recently changed their recommendation from Cyberduck (directions below), to Filezilla. They have made nice instructions found here .  We strongly recommend using Filezilla, as it will be supported by RC going forward. You can follow their instructions, except when you specify the host. You want the host to be an ncf workstation, such as ncfws??.rc.fas.harvard.edu, where the ?? is replaced by the number of the workstation you want to remotely login to. For choices, see here.

If you still want to continue with Cyberduck (which may be unreliable), you can follow the instructions below.

Cyberduck Instructions: Once installed, open the program up and from the File menu at the top, select “Open Connection”, and select SFTP at the top of the box that pops up.

 

In the Server box select your favorite workstation, or if you are just transferring a few files you can use ncflogin3 (See here for a list). Leave the Port at 22, enter your username and password, and click “Connect”. This will bring up something similar to Explorer on the PC or Finder on the Mac.

How do I look at and/or edit a text file on the NCF or my computer?

There are several ways to ‘read’ files from a workstation.

1. If you just want a quick view of the contents without opening up the file you can use ‘more’:

more <filename>

Press the space bar to advance the document, and ‘q’ to quit.

2. If you want to open the file up for editing or easier viewing, one of the nice text editors installed on the workstations is gedit.

gedit  <filename> &

This will open up a nice GUI based text editors that has all the bells and whistles like find/replace, go to line, and such. The & sign means to gedit in the background so that you can still type and do things in the terminal window you launched gedit from.

3. On your Mac, you also have several options. The one I recommend is Text Wrangler. Make sure to install the command line editor. You can use this to open any text file. From the Finder, by right clicking, selecting ‘Open With’, and then TextWrangler. The main reason I recommend textwrangler is because it has a powerful find/replace function, taking most unix commands. In addition, it has a command line tool, so that when you are in a terminal, you can open things right there, just like gedit on the workstation, via:

edit  <filename>

It will also let you edit ‘protected’ files by prompting you for a username and password, where other programs like the default text editor will often just deny you permission.

How do I change or reset my password?

To reset or change your password, use the Research Computing website here.

What if I want to VPN to the NCF from a different computer than usual?

The openauth token code required for the two-factor authentication is specific to your username. Therefore, if you want to VPN from a different machine, you will have to set this up again. An alternative is to get the openauth app for your iphone, blackberry, or android. The instructions for doing so are here.

How do I check how much space I have left?

From a workstation or an SSH session, type:

df –h <directory_you_want_to_check>

How do I compress my dicoms to save space?

From a workstation or an SSH session, go to the directory that contains all of your dicoms. Type:

tar -czvf name.tar.gz what_you_want_to_tar

For instance: tar -czvf mydicoms.tar.gz subjid* or tar -czvf mydicoms.tar.gz *.dcm

To uncompress them:  tar -xzvf name.tar.gz

How do I use the internet during VPN or from the workstations?

NCF system users will have access the Internet while connected to the NCF VPN, with any compatible web browser that will get an automated proxy configuration. For most browsers you should not have to do anything. You should also have access to your email. If this is not true please contact rchelp [at] fas [dot] harvard [dot] edu. There maybe some issues with the VPN if you have to access other remote secure networks simultaneously. In that case, please contact rchelp [at] fas [dot] harvard [dot] edu. You should also be able to access the internet from the workstations. If you launch firefox and get the message it is already running, see here If you experience any trouble accessing the internet, please contact rchelp [at] fas [dot] harvard [dot] edu.

How do I change the version of Matlab, Freesurfer, Afni, FSL or other brain analysis software packages?

We are in the process of switching how you choose which software packages you want. We were using 'list loaders', a set of scripts written in house (and described at the end of this FAQ). We are now transitioning to the 'module' system that is supported by Research Computing. They are also in the process of updating their system. We are only using their new system, or as they call it, the lmod module system. To see their more general notes, go here. See below for how to use them on the NCF.

To access the system, you need to either type a few lines in a terminal window, or put them at the beginning of your submit script, or add them to your .bashrc:

source new-modules.sh

module load ncf

The first line gets you on their new system, the second one loads the ncf specific modules.

To see the modules available to us:

module avail ncf

This will show a list that looks something like:

To select a module, for instance, a newer freesurfer:

module load freesurfer/5.3.0-ncf

If there is a module missing that you would like, for instance a matlab version, just email RC (rchelp [at] fas [dot] harvard [dot] edu).

See the instructions below for setting SPM, as there is an additional step.

The old system will continue to work for awhile, and is described below. But please start transitioning to the new system.

Old system

Thanks to Tim O'Keefe from the Neuroinfomatics group, we have a simple script to change between most of the major brain analysis software packages.  If there is a version you want not listed below, contact Tim (timothy_okeefe [at] harvard [dot] edu) and CBS help (cbssupport [at] neuroinfo [dot] org) for support.

For the following 'loader' scripts to work, you must be using the default enviornment (see here). 

To get a list of availabe versions, type:

list_loaders

This should give you something that looks like the list below. Which has the software package name followed by the available versions.  To select one, type the command (load_??) followed by the version:

load_fsl 4.1.7

or

load_freesurfer 5.1.0

IMPORTANT: If you want this change to be permenant, you need to add it to your .bashrc file (located in your home directory).  Add it below the area saying "Insert additional configuration below this line".

Changing SPM is a little more compliated, because your matlab path must be updated.  See here.

 

load_afni: 2011_05_26_1457
load_caret: 5.0
load_dcmtk: 3.6
load_freesurfer: 4.0.2  4.5.0
load_fsl: 4.0.3  4.1.6  4.1.7
load_java: 1.5  1.5.0_22  1.6  1.6.0_21_64  1.6.0_24
load_matlab: 7.11  7.4  7.7  7.8  7.9
load_mricron: 2009_12
load_nit: 1.0  1.0.1  1.0.2  1.1  1.2  1.2.1  1.2.2  1.3.2  1.3.3  1.3.4  1.4  1.4.1  1.5.0b  dev
load_php: 5.3.10
load_python: 2.7.0  2.7.2
load_sd: 1.5.1
load_spm: 2.0  5.0  8.0  8.1  8.4667
load_xjview: 8.4

 

How do I set or change my SPM version?

You can change your SPM version similar to any other brain analysis software using the tools described above.  However, you also need to add the SPM version to your matlab path.  You can do this by using one of the variables set in the default unix enviornment (see here), $_HVD_SPM_DIR.  To check what that variable is set to, type:

echo $_HVD_SPM_DIR

If the variable doesn't exist then you are not using the correct enviornment. If it is not the version you want, see how to change it via load_spm above.  Remember, if you want to use a different version of SPM on future logins and in different terminals, you must add it to your .bashrc file. Once you have the SPM you want you can add this variable to your ~/matlab/startup.m file (See here for editing files).

addpath(getenv('_HVD_SPM_DIR'));

This will prepend SPM and its subdirectories to your matlab path, which means it will supersede any version you had previously set in your path.  You should be aware that if you make changes to your path in matlab and then save it, this version of spm will become part of your 'permanent' path, as it was added on the fly at startup, but then you just saved the whole current thing.  This can become a problem if you are swiches versions back and forth.  This is also true of Freesurfer, who likes to add things to your matlab path, so make sure to check this occasionally and make sure mutliple things aren't added (via file - set path in matlab).

How do I streamline my SPM analysis?

A collaboration with IQSS (Alex Storer), a previous graduate student, Caitlin Carey, and our local Tim O'Keefe have resulted in a set of python scripts to help you with your SPM analysis. They are designed to take a batch script you create for a single subject, and apply it to a bunch of subjects. They start by downloading your data from CBSCentral. So even if you don't want to use them for analysis, you might want to use them for grabbing your data and converting to SPM niftis. The instructions for using them are located on GitHub. The scripts do not have to be downloaded, but you do have to load the module for them. In your script, or in your terminal window, you will need to have the following commands before you can use or even see them.

source new-modules.sh
module load ncf
module load cbs_spm/master-ncf

The first two lines might become optional, but are needed at the moment.

The instructions for use are located here: https://github.com/harvardinformatics/CBS_SPM/blob/master/README.org

Essentially, they replace the path information in your batch script with that of the new subject. You can easily check the new batch created, if you want to do anything nonstandard. They have been tested with basic preprocessing (including SPM12 where you can include slice timing information for SMS/multiband scans), through your first level analysis.

Firefox won’t open because it thinks it is already running.

The reason this happens is because Firefox uses one profile by default, and this profile gets “locked” when in use. To resolve this issues, when logged into workstation, open a terminal session and type in the command line:

firefox -profilemanager

A “Firefox – Choose your profile ” dialogue box will come up. Click on “Profile” -> “next” -> fill in a description, such as “second”, then click “finish” Then highlight the new profile, by clicking on it once with the mouse, and click on “Start firefox” Firefox should start up now.

Gedit is generally misbehaving.

Most issues, like random warnings, not opening, or not copy pasting correctly can be solved by deleting your local gedit preference file.

rm -R .gnome2/gedit/*

Once on the NCF, I can’t start applications such as Firefox or Matlab.

There are a few possibilities:

1) Please make sure that you are either typing the full path, or that the application is in your path. To check if an application is in your path, type:

which <program_or_script_name>

If that returns a path for the application, then you shouldn’t need the whole path, see other possibilities below.

2) Please verify that your home directory has not reached it’s max quota. Go here for instructions on checking your space.

3) If you are remotely logged in, the problem could involve X11 forwarding. On the Mac, make sure that you used the –X or –Y flag when sshing (example). On the PC if you are using Putty, there should be an option to check ‘Enable X11 Forwarding).

4) Make sure you are sourcing the default environment that includes most packages and sets your path.  Go here to see what it should look like. 

4) If you are still having issues, email rchelp [at] fas [dot] harvard [dot] edu.

I tried to ssh to a workstation and it gave me a scary message about the token or IP address changing, what should I do?

The error message looks something like:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ The RSA host key for ncfws01.rc.fas.harvard.edu has changed, and the key for the corresponding IP address 10.242.38.51 has a different value. This could either mean that DNS SPOOFING is happening or the IP address for the host and its host key have changed at the same time. Offending key for IP in /Users/stephanie/.ssh/known_hosts:8 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that the RSA host key has just been changed. The fingerprint for the RSA key sent by the remote host is ee:bf:41:7d:3f:00:29:1a:8f:05:99:b2:51:28:a0:93. Please contact your system administrator. Add correct host key in /Users/stephanie/.ssh/known_hosts to get rid of this message. Offending key in /Users/stephanie/.ssh/known_hosts:16 RSA host key for ncfws01.rc.fas.harvard.edu has changed and you have requested strict checking. Host key verification failed.

To fix it, you need to edit the file it tells you: /Users/username/.ssh/known_hosts.

For instance, if you got this message when trying to ssh to ncfws01.rs.fas.harvard.edu, you would open up the requested file and delete the line containing information for that workstation (highlighted in gray below).

 

 

 

How do I submit jobs to the compute cluster?

There are several ways you can submit a job.

1. You can submit from any workstation.

2. You can submit from any SSH session to an individual workstation 

3. You can SSH to the NCF login node (ncflogin1, ncflogin2, or ncflogin3). Just keep in mind that the ncflogin node is only for submitting scripts. It should not be used for analyzing data or other computationally intensive jobs as it is a shared space. Those types of things should be done from individual workstations.

From any of locations you can submit a job via the bsub command.

bsub –q ncf  <script_name>

When the job is finished, you will receive an email notification along with all the output that would have normally been written to the screen. Importantly, after submitting, you can close the terminal and logout, the script will still be running in the background. If the script you want to submit is not in your path, such as when you make your own script to run, you need to make sure you give the full pathname to the script and that it is executable (chmod u+x <filename>). If it is a standard script like procfast or recon-all this is not necessary. See here for how to submit a matlab script.

If the script takes arguments afterwards, such as recon-all, then enclose it in quotes:

bsub –q ncf “recon-all –all –subjid mysubject”

Other useful flags inlcude:

-J a_job_name (useful for giving your job a name that is easily identifiable)

-o output_filename (stores output of your script to a file, but will no longer email you when the script is done.  To get an email still, add a -N flag. You will still get an email when the script is done but the output from the script will be stored in the file specified by -o)

-e error_filename (stores any error messages generated by system.  Can be useful for RC to debug issues)

-m hostname (to choose an unbusy node to submit your work to.  To choose a host look here - you MUST be on the VPN, the nodes start with ncfc)

-n number of processors (This is the number of processors your submit job will go to, the default is 1.  Please be respectful of other users and don't hog the system.  To see how busy the compute cluster is, go here - you MUST be on the VPN.)

bsub –q ncf -J recon_s102 -0 s102_output -e s102_erroutput -m ncfc21 “recon-all –all –subjid mysubject”


Useful commands to work with submitted scripts (bsub tools).

Some useful commands:

bjobs

Displays all your jobs currently running.

bkill <job id>

This will kill a submitted job. You can find the job id by running bjobs, or it is given when you submit the job.

bpeek  <job id>

This allows you to ‘peek’ at the output of your job while it is running, to see what would have thus far been normally printed to the screen. This will not work if you directed the output of your job to a file with the –o opition, in that case you can look at the output file.

bsub –q ncf –o <output_filename> <script_name>

The –o option will redirect the standard output, or what would have been normally written to the screen if not submitting the job, to the filename given. In this case, you won’t receive an email when the job is done.