LHD Package

LHD Package

Data acquisition for LHD.

LHD_utils Module

pyfusion.acquisition.LHD.LHD_utils.get_free_bytes(path)

pyfusion.acquisition.LHD.LHD_utils.purge_old(path, wild, number=20)

acq Module

LHD acquisition.

class pyfusion.acquisition.LHD.acq.LHDAcquisition(config_name=None, **kwargs)

Bases: pyfusion.acquisition.base.BaseAcquisition

evaluate_other Module

fetch Module

LHD data fetchers. This gets the data directly from the data server, and so only runs on LHD dmana v0: Large chunks of code copied from Boyd, not covered by unit tests. v1:

See the docstring under class LHDTImeseries

class pyfusion.acquisition.LHD.fetch.LHDBaseDataFetcher(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.base.BaseDataFetcher

error_info(step=None)

can only access items that are part of self - others may be volatile

class pyfusion.acquisition.LHD.fetch.LHDIgetfileReader(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.LHD.fetch.LHDBaseDataFetcher

This uses the igetfile function to return one diagnostic at a time, on its own timebase. The original use of get_basic_diagnostics was to get a bunch of diags, on a given timebase. Might be better to separate the functions in the future. Will probably drop the dictionary approach and make each item a separte enetity in the .cfg file soon.

do_fetch()

class pyfusion.acquisition.LHD.fetch.LHDTimeseriesDataFetcher(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.LHD.fetch.LHDBaseDataFetcher

need: export Retrieve=~/retrieve/bin/ # (maybe not) export INDEXSERVERNAME=DasIndex.LHD.nifs.ac.jp/LHD

Debugging

Off-site in pyfusion:

# set the config to use LHD fetcher
pyfusion.config.set('DEFAULT','LHDfetcher','pyfusion.acquisition.LHD.fetch.LHDTimeseriesDataFetcher')
# choose a shot that doesn't exist locally
run pyfusion/examples/plot_signals.py shot_number=999 diag_name='VSL_6' dev_name='LHD'

On-site test lines for exes:

retrieve SX8O 74181 1 33
retrieve Magnetics3lab1 74181 1 33
2015: retrieve_t seems to only work on FMD
retrieve_t FMD 117242 1 33
different error messages on Magnetics3lab1

Using retrieve_t:

Don't know when it is needed -  always trying it first?
if it gives an error, calculate according to .prm
timeit fmd=retriever.retrieve('Magnetics3lab1',105396,1,[33],False)
142ms without retrieve_t, 224 with, including failure (set True in above)

do_fetch()

pyfusion.acquisition.LHD.fetch.LHD_A14_clk(shot)

Helper routine to fix up the undocumented clock speed changes in the A14

pyfusion.acquisition.LHD.fetch.persevere_with(cmd, attempts=2, quiet=False)

pyfusion.acquisition.LHD.fetch.read_data_from_file(fetcher)

pyfusion.acquisition.LHD.fetch.read_prm_file(filename, skip=1)

Read a prm file into a dictionary. Main entry point is via filename, possibly reserve the option to access via shot and subshot >>> pd = read_prm_file(filename=filename) >>> pd[‘Resolution(bit)’] [‘12’, ‘4’] skip=0 for .tprm files, which omit the ‘digitiser’ token

This comes from the line Aurora14,Resolution(bit),12,4 where (maybe?) last digit is 1: string, 4: mediumint, 5: float, 6 signed int, 7, bigint??

pyfusion.acquisition.LHD.fetch.retrieve_to_file(diagg_name=None, shot=None, subshot=None, channel=None, outdir=None, get_data=True)

run the retrieve standalone program to get data to files, and/or extract the parameter and summary information.

Retrieve Usage from Oct 2009 tar file: Retrieve DiagName ShotNo SubShotNo ChNo [FileName] [-f FrameNo] [-h TransdServer] [-p root] [-n port] [-w|–wait [-t timeout] ] [-R|–real ]

fetch_local Module

LHD data fetchers. Should REALLY use the code in base.py Large chunks of code copied from Boyd, not covered by unit tests, then copied back to this version to read .npz local data. THis is different to reading the files obtained by retrieve

class pyfusion.acquisition.LHD.fetch_local.LHDBaseDataFetcher(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.base.BaseDataFetcher

class pyfusion.acquisition.LHD.fetch_local.LHDTimeseriesDataFetcher(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.LHD.fetch_local.LHDBaseDataFetcher

do_fetch()

fetch_local_tmp Module

LHD data fetchers. Large chunks of code copied from Boyd, not covered by unit tests, then copied back to this version to read .npz local data. THis is different to reading the files obtained by retrieve

class pyfusion.acquisition.LHD.fetch_local_tmp.LHDBaseDataFetcher(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.base.BaseDataFetcher

class pyfusion.acquisition.LHD.fetch_local_tmp.LHDTimeseriesDataFetcher(acq, shot, config_name=None, **kwargs)

Bases: pyfusion.acquisition.LHD.fetch_local_tmp.LHDBaseDataFetcher

do_fetch()

pyfusion.acquisition.LHD.fetch_local_tmp.newload(filename, verbose=1)

Intended to replace load() in numpy

get_basic_diagnostics Module

Originally a script in example (called get_basic_params - still there get the basic plasma params for a given shot and range of times interpolation overhead only begins at 10k points, doubles time at 1Million!!

pyfusion.acquisition.LHD.get_basic_diagnostics.get_basic_diagnostics(diags=None, file_info={'R_ax': {'name': 'MagneticAxis', 'format': 'lhd_summary_data.csv'}, 'n_e19dL5': {'comment': 'mid_radius ne_dL, not nebar', 'name': 'average:nL\\(4029|3399\\)', 'format': 'fircall@{0}.dat'}, 'NBI1Pwr': {'name': 'NBI1Power', 'format': 'lhd_summary_data.csv'}, 'dw_pdt2': {'name': 'ddt2:Wp$', 'format': 'wp@{0}.dat'}, 'dw_pdt': {'name': 'ddt:Wp$', 'format': 'wp@{0}.dat'}, 'NBI': {'name': 'sum:NBI.*.Iacc.', 'format': 'nbi@{0}.dat'}, 'n_e19b0': {'comment': 'central line avg density', 'name': 'ne_bar\\(3[0-9]+\\)', 'format': 'fircall@{0}.dat'}, 'beta': {'name': '<beta-dia>', 'format': 'wp@{0}.dat'}, 'Gamma': {'name': 'GAMMA', 'format': 'lhd_summary_data.csv'}, 'Quad': {'name': 'Quadruple', 'format': 'lhd_summary_data.csv'}, 'b_0': {'name': 'MagneticField', 'format': 'lhd_summary_data.csv'}, '<n_e19>': {'name': 'ne_bar\\(3[0-9]+\\)', 'format': 'fircall@{0}.dat'}, 'ech': {'name': 'sum:.*', 'format': 'ech@{0}.dat'}, 'NBI2Pwr': {'name': 'NBI2Power', 'format': 'lhd_summary_data.csv'}, 'w_p': {'name': 'Wp$', 'format': 'wp@{0}.dat'}, 'di_pdt': {'name': 'ddt:Ip$', 'format': 'ip@{0}.dat'}, 'i_p': {'name': 'Ip$', 'format': 'ip@{0}.dat'}}, shot=54196, times=None, delay=None, exception=False, debug=0)

return a list of np.arrays of normally numeric values for the

times given, for the given shot. Will access server if env(‘IGETFILE’) points to an exe, else accesses cache This is the first version to specifically allow for access through pyfusion.cfg There are two types of access:

I/ single diag on its own timebase II/ the original multi diag on a given timebase (i.e. that from flucstrcs)

Stage 1 puts the file_info into .cfg file just for I/ single diag access. Ideally the file_info for II/ sho;d be in .cfg also. For stage I/, we call it with a file_info dict constructed on the spot as a dictionary with one just entry (for diags[0]). Some diagnostics are

R_ax, n_e19dL5, NBI1Pwr, dw_pdt2, dw_pdt, NBI, n_e19b0, beta, Gamma, Quad, b_0, <n_e19>, ech, NBI2Pwr, w_p, di_pdt, i_p

pyfusion.acquisition.LHD.get_basic_diagnostics.get_delay(shot)

pyfusion.acquisition.LHD.get_basic_diagnostics.get_flat_top(shot=54196, times=None, smooth_dt=None, maxddw=None, hold=0, debug=0)

debug=1 gives a plot

pyfusion.acquisition.LHD.get_basic_diagnostics.localigetfilepath = '/home/bdb112/data/datamining/cache//'

Make a list of diagnostics, and how to obtain them, as a dictionary of dictionaries: Top level keys are the short names for the dignostics Next level keys are

format: format string to generate file name from shot. name: wild card name to find in the file header

For the case of static parameters, the ‘format’ field is a csv file containing them all.

import_LHD_summary_csv Module

make_summary_egdb Module

read_igetfile Module

pyfusion.acquisition.LHD.read_igetfile.call_igetfile(path_to_igetfile, filename)

run the igetfile program, returning the filename, which can be used for later deleting. 2017 - separate into folders per diag. Assume filename looks like somedir/wp/wp@90000.dat

class pyfusion.acquisition.LHD.read_igetfile.igetfile(filename=None, fileformat=None, shot=None, verbose=0, plot=False, debug=0, hold=True)

read in data, optionally plot (True gets default channel, N gets ch=N if filename contains a {, assume it is a format let get_basic_params deal with .bz2 files etc for now.

data = None

filename = ''

plot(hold=0, ch=-1, tstart=None, tend=None, tstep=None, scl=None, navg=1, debug=1, axes=None, *args, **kwargs)

ch=-1 -> Plot all the data items - can take a long time.

shot = None

vardict = {}

pyfusion.acquisition.LHD.read_igetfile.read_igetfile(filename=None, verbose=0, plot=True, hold=True, debug=0, quiet=0)

Read LHD data from igetfile Note that the plot option on this call is crude, compared to in igetfile

pseudocode:

read file into a buffer find “[Parameters]” find “[data]” put brackets around lines with = sign and unquoted comma read stuff between read data into a rank 2 array

( This example creates it’s own file-like object - not normally required - also note that we use n (double backslash. Finally, this test runs into a problem at the first line of data maybe new since 2.7?)

>>> from StringIO import StringIO
>>> data = ' \n'.join(["%f, %f, %f" % (n/1e3, sin(n/1e3), cos(n/1e3)) for n in range(4)])
>>> st=StringIO("#[Parameters]\n# Name = 'mmw'\nShotNo=50623\n# Dimno=1 \n "         + "# DimName = 'Time'\n# DimSize = 2600\n# DimUnit = 's'\n# \n"         + "# ValNo = 2\n# ValName = 'nL()', 'nL()'\n# ValUnit = 'e19m-2', 'e19m-2'\n"         + "# [Comments]\n#  \n# [data]\n"         + data+" \n ")
>>> read_igetfile(st)[0][2]
array([ 0.002   ,  0.002   ,  0.999998])

tests Module

Test code for LHD data acquision.

class pyfusion.acquisition.LHD.tests.TestLHDDataAcq

Bases: pyfusion.test.tests.PfTestBase

lhd = True

test_return_type()