Submit
Path:
~
/
/
proc
/
thread-self
/
root
/
opt
/
alt
/
python35
/
lib
/
python3.5
/
site-packages
/
joblib
/
__pycache__
/
File Content:
numpy_pickle_utils.cpython-35.pyc
��X�X � @ s� d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l m Z m Z m Z m Z y d d l m Z Wn" e k r� d d l m Z Yn Xe r� e j Z e j Z e Z n e j Z e j Z y d d l Z Wn e k r.d Z Yn Xy d d l Z Wn e k rYd Z Yn Xd Z d Z d Z d Z d Z d Z! d d d d d f Z" e j# e j$ g Z% e d k r�e% j& e j' � e( d d � e e e e e! f D� � Z) d d Z* d d � Z+ d d � Z, d d � Z- e j. d d d � � Z/ d d f d d � Z0 d Z1 d Z2 d Z3 d Z4 d! Z5 Gd" d# � d# e j6 � Z7 Gd$ d% � d% e7 � Z8 d d& Z9 d' d( d) � Z: d S)*zFUtilities for fast persistence of big data, with optional compression.� N)�closing� )� PY3_OR_LATER�PY26�PY27� _basestring)�RLocks ZFs xs �s BZs �7zXZs ] �zlib�bz2�lzma�xz�gzipc c s | ] } t | � Vq d S)N)�len)�.0�prefix� r �/numpy_pickle_utils.py� <genexpr>: s r i � c C s� | j d � | j t � } | j d � | j t � r< d S| j t � rO d S| j t � rb d S| j t � ru d S| j t � r� d S| j t � r� d Sd S) z�Return the compressor matching fileobj. Parameters ---------- fileobj: file object Returns ------- str in {'zlib', 'gzip', 'bz2', 'lzma', 'xz', 'compat', 'not-compressed'} r r r r r r �compatznot-compressed) �seek�read�_MAX_PREFIX_LEN� startswith� _ZLIB_PREFIX� _GZIP_PREFIX� _BZ2_PREFIX� _LZMA_PREFIX� _XZ_PREFIX� _ZFILE_PREFIX)�fileobjZ first_bytesr r r �_detect_compressorD s r! c C s9 t s t r" t | t j � r" | St j | d t �Sd S)z0Return a buffered version of a read file object.� buffer_sizeN)r r � isinstancer �BZ2File�io�BufferedReader�_IO_BUFFER_SIZE)�fobjr r r �_buffered_read_filed s r) c C s? t s t r( t | t j � r( t | � St j | d t �Sd S)z1Return a buffered version of a write file object.r" N) r r r# r r$ r r% �BufferedWriterr' )r( r r r �_buffered_write_fileo s r+ c c s� t | � } t | t t � � r- | j j } | d k r[ t j d | t d d �| Vn�| t k s| t | t t � � r� | d k r� t j d t � d d �t | t j � r� | d k r� t j d t � d d �t | t t � � r� | Vn� | d k rt t | d � � Vn� | d k r=t t | d � � Vn� | d k r�t rit t j | d � � Vq�t t j | j d � � Vnl | d k s�| d k r�t d k r�t t j | d � � Vq�t d j t j d t j d � � � n | Vd S)a- Utility function opening the right fileobject from a filename. The magic number is used to choose between the type of file object to open: * regular file object (default) * zlib file object * gzip file object * bz2 file object * lzma file object (for xz and lzma compressor) Parameters ---------- fileobj: file object compressor: str in {'zlib', 'gzip', 'bz2', 'lzma', 'xz', 'compat', 'not-compressed'} filename: str filename path corresponding to the fileobj parameter. mmap_mode: str memory map mode that should be used to open the pickle file. This parameter is useful to verify that the user is not trying to one with compression. Default: None. Returns ------- a file like object r zjThe file '%s' has been generated with a joblib version less than 0.10. Please regenerate this pickle file.� stacklevelr Nz�File "%(filename)s" is compressed using "%(compressor)s" which is not compatible with mmap_mode "%(mmap_mode)s" flag passed. mmap_mode option will be ignored.zuIn memory persistence is not compatible with mmap_mode "%(mmap_mode)s" flag passed. mmap_mode option will be ignored.r �rbr r r r zHLzma decompression is not available for this version of python ({0}.{1})r r )r! r# �tuple�_COMPRESSOR_CLASSES� __class__�__name__�warnings�warn�DeprecationWarning� _COMPRESSORS�localsr% �BytesIOr) �BinaryZlibFile�BinaryGzipFiler r r$ �namer �LZMAFile�NotImplementedError�format�sys� version_info)r �filenameZ mmap_modeZ compressorr r r �_read_fileobject| sB rA � c C s� | d } | d } | d k r9 t t | d d | �� S| d k ra t t j | d d | �� St d k r� | d k r� t t j | d d t j d | �� St d k r� | d k r� t t j | d d | d t j �� St t | d d | �� Sd S) z6Return the right compressor file object in write mode.r r r �wb� compresslevelr Nr �check�presetr r= ) r+ r9 r r$ r r; � CHECK_NONE� FORMAT_ALONEr8 )r@ �compressZcompressmethodrD r r r �_write_fileobject� s$ rJ i c @ s' e Z d Z d Z e j Z d d d d � Z d d � Z e d d � � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d d � Z d d d � Z d0 d" d# � Z d$ d% � Z d& d' � Z d( d) � Z d* d+ d, � Z d- d. � Z d/ S)1r8 a� A file object providing transparent zlib (de)compression. A BinaryZlibFile can act as a wrapper for an existing file object, or refer directly to a named file on disk. Note that BinaryZlibFile provides only a *binary* file interface: data read is returned as bytes, and data to be written should be given as bytes. This object is an adaptation of the BZ2File object and is compatible with versions of python >= 2.6. If filename is a str or bytes object, it gives the name of the file to be opened. Otherwise, it should be a file object, which will be used to read or write the compressed data. mode can be 'rb' for reading (default) or 'wb' for (over)writing If mode is 'wb', compresslevel can be a number between 1 and 9 specifying the level of compression: 1 produces the least compression, and 9 (default) produces the most compression. r- � c C s t � | _ d | _ d | _ t | _ d | _ d | _ t | t � sf d | k o` d k n r{ t d j | � � � | d k r� t } t j | j � | _ d | _ d | _ nO | d k r� t } t j | t j | j t j d � | _ n t d | f � � t | t � r<t | | � | _ d | _ | | _ n? t | d � sZt | d � ro| | _ | | _ n t d � � d S)NFr r rK zFcompresslevel must be between an integer between 1 and 9, you gave {0}r- � rC zInvalid mode: %rTr �writez1filename must be a str or bytes object, or a file���) r �_lock�_fp�_closefp� _MODE_CLOSED�_mode�_pos�_sizer# �int� ValueErrorr= � _MODE_READr � decompressobj�wbits� _decompressor�_buffer�_buffer_offset� _MODE_WRITE� compressobj�DEFLATED� DEF_MEM_LEVEL� _compressorr �open�hasattr� TypeError)�selfr@ �moderD � mode_coder r r �__init__ s< - zBinaryZlibFile.__init__c C s� | j �� | j t k r d SzV | j t t f k rA d | _ n1 | j t k rr | j j | j j � � d | _ Wd z | j r� | j j � Wd d | _ d | _ t | _ d | _ d | _ XXWd QRXd S)z�Flush and close the file. May be called more than once without error. Once the file is closed, any other operation on it will raise a ValueError. NFrL r )rO rS rR rX �_MODE_READ_EOFr[ r^ rP rM rb �flushrQ �closer\ r] )rf r r r rl 9 s" zBinaryZlibFile.closec C s | j t k S)z True if this file is closed.)rS rR )rf r r r �closedS s zBinaryZlibFile.closedc C s | j � | j j � S)z3Return the file descriptor for the underlying file.)�_check_not_closedrP �fileno)rf r r r ro X s zBinaryZlibFile.filenoc C s | j � o | j j � S)z)Return whether the file supports seeking.)�readablerP �seekable)rf r r r rq ] s zBinaryZlibFile.seekablec C s | j � | j t t f k S)z/Return whether the file was opened for reading.)rn rS rX rj )rf r r r rp a s zBinaryZlibFile.readablec C s | j � | j t k S)z/Return whether the file was opened for writing.)rn rS r^ )rf r r r �writablef s zBinaryZlibFile.writablec C s] | j rY t | j d d � } d } | d k rC | d j | � 7} | d 7} t | � � d S)Nr: z I/O operation on closed filez {0}�.)rm �getattrrP r= rW )rf Zfname�msgr r r rn m s z BinaryZlibFile._check_not_closedc C s2 | j t t f k r. | j � t j d � � d S)NzFile not open for reading)rS rX rj rn r% �UnsupportedOperation)rf r r r �_check_can_readv s z BinaryZlibFile._check_can_readc C s, | j t k r( | j � t j d � � d S)NzFile not open for writing)rS r^ rn r% rv )rf r r r �_check_can_write{ s z BinaryZlibFile._check_can_writec C sP | j t t f k r. | j � t j d � � | j j � sL t j d � � d S)Nz3Seeking is only supported on files open for readingz3The underlying file object does not support seeking)rS rX rj rn r% rv rP rq )rf r r r �_check_can_seek� s z BinaryZlibFile._check_can_seekc C s� | j t k r d Sx� | j t | j � k r� y. | j j pL | j j t � } | s[ t � Wn+ t k r� t | _ | j | _ d SYn X| j j | � | _ d | _ q Wd S)NFr T)rS rj r] r r\ r[ � unused_datarP r � _BUFFER_SIZE�EOFErrorrT rU � decompress)rf �rawblockr r r � _fill_buffer� s zBinaryZlibFile._fill_bufferTc C s� | j | j d � | _ d | _ g } xG | j � rq | rM | j | j � | j t | j � 7_ d | _ q+ W| r� d j | � Sd S)Nr rL )r\ r] r �appendrT r �join)rf � return_data�blocksr r r � _read_all� s zBinaryZlibFile._read_allc C s@ | j | } | t | j � k rd | j | j | � } | | _ | j t | � 7_ | r` | Sd S| j | j d � | _ d | _ g } x� | d k r(| j � r(| t | j � k r� | j d | � } | | _ n | j } d | _ | r | j | � | j t | � 7_ | t | � 8} q� W| r<d j | � Sd S)Nr rL )r] r r\ rT r r� r� )rf Zn_bytesr� �end�datar� r r r � _read_block� s* zBinaryZlibFile._read_blockr c C sR | j �B | j � | d k r$ d S| d k r: | j � S| j | � SWd QRXd S)z�Read up to size uncompressed bytes from the file. If size is negative or omitted, read until EOF is reached. Returns b'' if the file is already at EOF. r rL N)rO rw r� r� )rf �sizer r r r � s zBinaryZlibFile.readc C s( | j � t j j | | � SWd QRXd S)z_Read up to len(b) bytes into b. Returns the number of bytes read (0 for EOF). N)rO r% �BufferedIOBase�readinto)rf �br r r r� � s zBinaryZlibFile.readintoc C s� | j �r | j � t r6 t | t � r6 | j � } | j j | � } | j j | � | j t | � 7_ t | � SWd QRXd S)z�Write a byte string to the file. Returns the number of uncompressed bytes written, which is always len(data). Note that due to buffering, the file on disk may not reflect the data written until close() is called. N) rO rx r r# � memoryview�tobytesrb rI rP rM rT r )rf r� � compressedr r r rM � s zBinaryZlibFile.writec C sP | j j d d � t | _ d | _ t j | j � | _ d | _ d | _ d S)Nr rL ) rP r rX rS rT r rY rZ r[ r\ r] )rf r r r �_rewind� s zBinaryZlibFile._rewindr c C s� | j �� | j � | d k r# nj | d k r? | j | } nN | d k rz | j d k rj | j d d � | j | } n t d | f � � | | j k r� | j � n | | j 8} | j | d d �| j SWd QRXd S)a� Change the file position. The new position is specified by offset, relative to the position indicated by whence. Values for whence are: 0: start of stream (default); offset must not be negative 1: current stream position 2: end of stream; offset must not be positive Returns the new file position. Note that seeking is emulated, so depending on the parameters, this operation may be extremely slow. r r r r� Fz Invalid value for whence: %sN)rO ry rT rU r� rW r� r� )rf �offset�whencer r r r s zBinaryZlibFile.seekc C s& | j � | j � | j SWd QRXd S)z!Return the current file position.N)rO rn rT )rf r r r �tell* s zBinaryZlibFile.tellNrN ) r1 � __module__� __qualname__�__doc__r � MAX_WBITSrZ ri rl �propertyrm ro rq rp rr rn rw rx ry r r� r� r r� rM r� r r� r r r r r8 � s, ) *r8 c @ s e Z d Z d Z d Z d S)r9 a A file object providing transparent gzip (de)compression. If filename is a str or bytes object, it gives the name of the file to be opened. Otherwise, it should be a file object, which will be used to read or write the compressed data. mode can be 'rb' for reading (default) or 'wb' for (over)writing If mode is 'wb', compresslevel can be a number between 1 and 9 specifying the level of compression: 1 produces the least compression, and 9 (default) produces the most compression. � N)r1 r� r� r� rZ r r r r r9 1 s r9 � zran out of datac C s� t � } xh yL | j | t | � � } | | 7} t | � d k sV t | � | k rW PWq t j k ro Yq Xq Wt | � | k r� d } t | | | t | � f � � n | Sd S)a Read from file-like object until size bytes are read. Raises ValueError if not EOF is encountered before size bytes are read. Non-blocking objects only supported if they derive from io objects. Required as e.g. ZipExtFile in python 2.6 can return less data than requested. This function was taken from numpy/lib/format.py in version 1.10.2. Parameters ---------- fp: file-like object size: int error_template: str Returns ------- a bytes object The data read in bytes. r z)EOF: reading %s, expected %d bytes got %dN)�bytesr r r% �BlockingIOErrorrW )�fpr� Zerror_templater� �rru r r r � _read_bytesH s $ "r� );r� �pickler> r% r r r r2 � contextlibr Z_compatr r r r � threadingr � ImportError�dummy_threading� _Unpickler� Unpickler�_Pickler�Pickler�range�xrangeZnumpyZnpr r r r r r r r5 �GzipFiler$ r/ r� r; �maxr r' r! r) r+ �contextmanagerrA rJ rR rX rj r^ r{ r� r8 r9 � BUFFER_SIZEr� r r r r �<module> st " Y� ;
Edit
Rename
Chmod
Delete
FILE
FOLDER
Name
Size
Permission
Action
__init__.cpython-35.opt-1.pyc
4779 bytes
0644
__init__.cpython-35.pyc
4779 bytes
0644
_compat.cpython-35.opt-1.pyc
677 bytes
0644
_compat.cpython-35.pyc
677 bytes
0644
_memory_helpers.cpython-35.opt-1.pyc
3223 bytes
0644
_memory_helpers.cpython-35.pyc
3223 bytes
0644
_multiprocessing_helpers.cpython-35.opt-1.pyc
922 bytes
0644
_multiprocessing_helpers.cpython-35.pyc
922 bytes
0644
_parallel_backends.cpython-35.opt-1.pyc
12908 bytes
0644
_parallel_backends.cpython-35.pyc
12908 bytes
0644
disk.cpython-35.opt-1.pyc
2571 bytes
0644
disk.cpython-35.pyc
2571 bytes
0644
format_stack.cpython-35.opt-1.pyc
9075 bytes
0644
format_stack.cpython-35.pyc
9123 bytes
0644
func_inspect.cpython-35.opt-1.pyc
9763 bytes
0644
func_inspect.cpython-35.pyc
9763 bytes
0644
hashing.cpython-35.opt-1.pyc
6653 bytes
0644
hashing.cpython-35.pyc
6653 bytes
0644
logger.cpython-35.opt-1.pyc
4184 bytes
0644
logger.cpython-35.pyc
4184 bytes
0644
memory.cpython-35.opt-1.pyc
28122 bytes
0644
memory.cpython-35.pyc
28122 bytes
0644
my_exceptions.cpython-35.opt-1.pyc
2750 bytes
0644
my_exceptions.cpython-35.pyc
2750 bytes
0644
numpy_pickle.cpython-35.opt-1.pyc
15536 bytes
0644
numpy_pickle.cpython-35.pyc
15536 bytes
0644
numpy_pickle_compat.cpython-35.opt-1.pyc
7262 bytes
0644
numpy_pickle_compat.cpython-35.pyc
7393 bytes
0644
numpy_pickle_utils.cpython-35.opt-1.pyc
16434 bytes
0644
numpy_pickle_utils.cpython-35.pyc
16434 bytes
0644
parallel.cpython-35.opt-1.pyc
25898 bytes
0644
parallel.cpython-35.pyc
25898 bytes
0644
pool.cpython-35.opt-1.pyc
18841 bytes
0644
pool.cpython-35.pyc
18841 bytes
0644
testing.cpython-35.opt-1.pyc
2657 bytes
0644
testing.cpython-35.pyc
2657 bytes
0644
N4ST4R_ID | Naxtarrr
