Cross package maker. DEB/RPM generation or conversion. Derived from jordansissel/fpm.

⌈⌋ branch:  cross package maker


Check-in [d1be6d8c51]

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Basic inheritance for =target specifiers. Make meta extractor not consume mixed *|#|// comment prefixes. Clean up pack list from empty elements. Move copying logic out of main function.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:d1be6d8c515ff29ec4d8ae25dafd09b545df9540
User & Date: mario 2015-02-06 18:07:50
Context
2015-02-06
20:52
Make some inherited relative target paths working. check-in: bc0c714540 user: mario tags: trunk
18:07
Basic inheritance for =target specifiers. Make meta extractor not consume mixed *|#|// comment prefixes. Clean up pack list from empty elements. Move copying logic out of main function. check-in: d1be6d8c51 user: mario tags: trunk
2015-01-28
12:51
Fix pkg.provides += [] addition. Strip any VCS #hashversionsuffixes from version strings. check-in: e08478f4fe user: mario tags: trunk
Changes

Changes to lib/fpm/package/src.rb.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
..
97
98
99
100
101
102
103




104




105
106




107
108

109
110
111
112
113
114
115
116

117
118





119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153

154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170












171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193






194
195
196
197
198
199
200
...
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
# encoding: ascii
# api: fpm
# title: generic script source
# description: Utilizes `pack:` specifier from meta/comment block in script files
# type: package
# category: source
# version: 0.7
# state: beta
# architecture: all
# license: MITL
# author: mario#include-once:org
# config: <arg name="src-only" description="Only apply main pack: spec. Don't scan recursively.">
# depends:
# pack: src.rb, README=README.txt, src/*.png
................................................................................
    #   rel/src1 => { src2=>dest, src3=>dest } ,
    #   rel/src2 => { src4=> .. }
    # }
    @map = {:spec_start => {path => path}}
    ::Dir.chdir(attributes[:chdir] || ".") do
      rewrite_map(path)
      logger.debug(@map)









      # Copy all referenced files.
      copied = {}




      @map.each do |from,pack|
        pack.each do |src,dest|

          # Each file can hold a primary declaration and override its target filename / make itself absent.
          if dest and @map[dest] and @map[dest][dest]
            dest = @map[dest][dest]
          end
          # References above the src/.. would also cross the staging_dir, give warning
          if dest and dest.match(/^\.\.\//)
            logger.warn("Referencing above ../ the basedir from #{from}")
          end

          # Else just apply pack{} map copying verbatim, possibly overwriting or duplicating files.
          if not copied[src+">"+dest]





            real_copy("#{src}", "#{staging_path}/#{attributes[:prefix]}/#{dest}") if dest != ""
            copied[src+">"+dest] = 1
          end
        end
      end
    end
  end


  protected

  
  # Scan files for file,src=dest mapping lists.
  #
  # Each source file can specify references as `pack: two.sh, sub/three.py`
  # where each scripts´ subdirectory becomes the next base directory.
  #
  # The `dest` parameter is carried over from the previous old=new
  # filename pack{} map.
  #
  def rewrite_map(from, dest="")

    # lazy workaround to prevent neverending loops/references
    return if @map[from]
    
    # check if file is present, get meta data
    if not File.exists?(from)
      logger.warn("Skipping non-existent #{from} file")
      return
    elsif File.directory?(from)
      @map[from] = {}
      return
    else 
      packlist = get_meta(from)["pack"]  # adding [splitfn(from)[1]] itself would override renames
    end

    
    # relative dir and basename
    dir, fn = splitfn(from)   # fpm/ src.rb

    # @example
    #   from = fpm/src.rb
    #          `pack: exe.rb, ../README, foo=bar, test/*`
    # @result
    #   pack[fpm/src.rb] = fpm/exe.rb=>fpm/exe.rb, README=>README, fpm/foo=>fpm/bar, fpm/test/123=>...}

    # iterate over listed references
    @map[from] = pack = {}
    packlist.each do |fnspec|

      # specifier can be one of: `src, src=dest, src*, dir/src*=dest/`
      src, dest = fnspec.split("=", 2) # dest can be nil (=use src basename), or empty "" (=skip file)













      # glob expansion      
      if src.match(/[\[\{\*\?]/)
        src = ::Dir.glob(dir + src)
        logger.warn("Nothing matched for '#{fnspec}' in '#{from}'") if src.empty?
      else
        src = [dir + src]
      end

      # combine with dest, and recursively scan each referenced source file
      src.each do |src|
        pack[consolidate(src)] = consolidate(dest ? dir+dest : src)
        rewrite_map(consolidate(src)) if not @attributes[:src_only?]
      end

    end
  end # def rewrite_map


  # split dir/name/ from basename 
  def splitfn(path)
    path =~ /\A  ((?: .*\/ )?)  ([^\/]+)  \Z/x
    return [$1, $2]
  end







  
  # remove ./ and dir/../ segments
  def consolidate(path)
    path = path.gsub(/(?<=^|\/)\.\//, "")    # strip out "./" self-referenting dirs
    path = path.gsub(/(?<=^|\/)(?!\.\.\/)[\w.-]+\/\.\.\//, "")    # consolidate "sub/../" relative paths
  end
................................................................................

    # read file (first 8K would suffice)
    src = File.read(path, 1<<13, 0, :encoding=>'ASCII-8BIT')
    fields = { "pack" => "" }

    # extract first comment block,
    if src and src =~ /( \/\*+ .+? \*\/ | (?:[ \t]* \* | \#(?!!) [^\n]*\n)+ )/mix
      src = $1.gsub(/^[ \t\*\/\#]+/, "").to_s # does not honor indentation

      # split meta block from comment (on first empty line)
      src, fields["comment"] = src.split(/\r?\n\r?\n/, 2)  # eh, Ruby, Y U NO PROVIDE \R ?
      # split key: value fields
      fields.merge!(Hash[
         src.to_s.scan(/^([\w-]+): [[:blank:]]* ([^\n]* (?:\n (?![\w-]+:) [^\n]+)* )/x).map{ |k,v| [k.downcase, v] }
      ])

      # use input basename or id: field as package name
      fields["id"] = fields["id"] || path.match(/([^\/]+?)(\.\w+)?\Z/)[1]
      # split up pack: comma-separated list, don't expand src=dest yet
#p fields
      fields["pack"] = fields["pack"].split(/(?<!\\)[,\s]+/)
    else 
      fields["pack"] = []
    end
    return fields
  end # def meta








|







 







>
>
>
>

>
>
>
>
|
<
>
>
>
>
|
|
>
|
|
|
|
|
|
|
|
>
|
|
>
>
>
>
>
|
|
|
|
|
|
<

<
<
<






|
|

|



<










>













|



>
>
>
>
>
>
>
>
>
>
>
>
|










|











>
>
>
>
>
>







 







|












|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
..
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142

143



144
145
146
147
148
149
150
151
152
153
154
155
156

157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
...
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
# encoding: ascii
# api: fpm
# title: generic script source
# description: Utilizes `pack:` specifier from meta/comment block in script files
# type: package
# category: source
# version: 0.8
# state: beta
# architecture: all
# license: MITL
# author: mario#include-once:org
# config: <arg name="src-only" description="Only apply main pack: spec. Don't scan recursively.">
# depends:
# pack: src.rb, README=README.txt, src/*.png
................................................................................
    #   rel/src1 => { src2=>dest, src3=>dest } ,
    #   rel/src2 => { src4=> .. }
    # }
    @map = {:spec_start => {path => path}}
    ::Dir.chdir(attributes[:chdir] || ".") do
      rewrite_map(path)
      logger.debug(@map)
      # And copie files to staging path.
      copy_by_map({})
    end
  end


  protected

  
  # Copy all referenced files.

  #
  #
  #
  def copy_by_map(copied)
    @map.each do |from,pack|
      pack.each do |src,dest|

        # Each file can hold a primary declaration and override its target filename / make itself absent.
        if dest and @map[dest] and @map[dest][dest]
          dest = @map[dest][dest]
        end
        # References above the src/.. would also cross the staging_dir, give warning
        if dest and dest.match(/^\.\.\//)
          logger.warn("Referencing above ../ the basedir from #{from}")
        end

        # Else just apply pack{} map copying verbatim, possibly overwriting or duplicating files.
        if not copied[src+">"+dest]
          # For absolute dest=/targets: omit the --prefix path
          unless (Pathname.new dest).absolute?
            dest = attributes[:prefix] + "/" + dest
          end

          real_copy("#{src}", "#{staging_path}/#{dest}") if dest != ""
          copied[src+">"+dest] = 1
        end
      end
    end
  end





  
  # Scan files for file,src=dest mapping lists.
  #
  # Each source file can specify references as `pack: two.sh, sub/three.py`
  # where each scripts´ subdirectory becomes the next base directory.
  #
  # TODO: The `destp` parameter is carried over from the patent old=new
  # filename pack{} map, should be joined in (doesn't work yet).
  #
  def rewrite_map(from, dest_p=nil)

    # lazy workaround to prevent neverending loops/references
    return if @map[from]

    # check if file is present, get meta data
    if not File.exists?(from)
      logger.warn("Skipping non-existent #{from} file")
      return
    elsif File.directory?(from)
      @map[from] = {}
      return
    else 
      packlist = get_meta(from)["pack"]  # adding [splitfn(from)[1]] itself would override renames
    end
puts "#{from}=#{dest_p} #pack= #{packlist}"
    
    # relative dir and basename
    dir, fn = splitfn(from)   # fpm/ src.rb

    # @example
    #   from = fpm/src.rb
    #          `pack: exe.rb, ../README, foo=bar, test/*`
    # @result
    #   pack[fpm/src.rb] = fpm/exe.rb=>fpm/exe.rb, README=>README, fpm/foo=>fpm/bar, fpm/test/123=>...}

    # iterate over listed references
    @map[from] = pack = {}
    packlist.each do |fnspec|
    
      # specifier can be one of: `src, src=dest, src*, dir/src*=dest/`
      src, dest = fnspec.split("=", 2) # dest can be nil (=use src basename), or empty "" (=skip file)

      # join eventual parent dest_p (module/two/) and new dest (sub/)
      if dest_p
        if (Pathname.new dest).absolute?
          dest
        elsif dest
          dest = consolidate(dest_p + dest)
        else
          dest ||= dest_p
        end
      end
puts "+ #{src}=#{dest}"

      # glob expansion
      if src.match(/[\[\{\*\?]/)
        src = ::Dir.glob(dir + src)
        logger.warn("Nothing matched for '#{fnspec}' in '#{from}'") if src.empty?
      else
        src = [dir + src]
      end

      # combine with dest, and recursively scan each referenced source file
      src.each do |src|
        pack[consolidate(src)] = consolidate(dest ? dir+dest : src)
        rewrite_map(consolidate(src), dironly(dest)) if not @attributes[:src_only?]
      end

    end
  end # def rewrite_map


  # split dir/name/ from basename 
  def splitfn(path)
    path =~ /\A  ((?: .*\/ )?)  ([^\/]+)  \Z/x
    return [$1, $2]
  end

  # get only dir/name/ until trailing slash from path (filename is optional)
  def dironly(path)
    path =~ /\A  ((?: .*\/ )?)  ([^\/]*)  \Z/x
    return $1
  end

  
  # remove ./ and dir/../ segments
  def consolidate(path)
    path = path.gsub(/(?<=^|\/)\.\//, "")    # strip out "./" self-referenting dirs
    path = path.gsub(/(?<=^|\/)(?!\.\.\/)[\w.-]+\/\.\.\//, "")    # consolidate "sub/../" relative paths
  end
................................................................................

    # read file (first 8K would suffice)
    src = File.read(path, 1<<13, 0, :encoding=>'ASCII-8BIT')
    fields = { "pack" => "" }

    # extract first comment block,
    if src and src =~ /( \/\*+ .+? \*\/ | (?:[ \t]* \* | \#(?!!) [^\n]*\n)+ )/mix
      src = $1.gsub(/^[ \t]*(\*+|\/+|\#+)[ \t]*/, "").to_s # does not honor indentation

      # split meta block from comment (on first empty line)
      src, fields["comment"] = src.split(/\r?\n\r?\n/, 2)  # eh, Ruby, Y U NO PROVIDE \R ?
      # split key: value fields
      fields.merge!(Hash[
         src.to_s.scan(/^([\w-]+): [[:blank:]]* ([^\n]* (?:\n (?![\w-]+:) [^\n]+)* )/x).map{ |k,v| [k.downcase, v] }
      ])

      # use input basename or id: field as package name
      fields["id"] = fields["id"] || path.match(/([^\/]+?)(\.\w+)?\Z/)[1]
      # split up pack: comma-separated list, don't expand src=dest yet
#p fields
      fields["pack"] = fields["pack"].split(/(?<!\\) (?!\s*=) [,\s]+ /x).map{|x|x.strip}.compact.reject{|x|x.empty?}
    else 
      fields["pack"] = []
    end
    return fields
  end # def meta