Remake
 All Classes Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages
remake.cpp
Go to the documentation of this file.
1 /**
2 @mainpage Remake, a build system that bridges the gap between make and redo.
3 
4 As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
5 named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
6 syntax:
7 
8 @verbatim
9 target1 target2 ... : prerequisite1 prerequisite2 ...
10  shell script
11  that builds
12  the targets
13 @endverbatim
14 
15 A target is known to be up-to-date if all its prerequisites are. If it
16 has no known prerequisites yet the file already exits, it is assumed to
17 be up-to-date. Obsolete targets are rebuilt thanks to the shell script
18 provided by the rule.
19 
20 As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
21 addition to these static dependencies. Whenever a script executes
22 <tt>remake prerequisite4 prerequisite5 ...</tt>, these prerequisites are
23 rebuilt if they are obsolete. (So <b>remake</b> acts like
24 <b>redo-ifchange</b>.) Moreover, all the dependencies are stored in file
25 <b>.remake</b> so that they are remembered in subsequent runs. Note that
26 dynamic dependencies from previous runs are only used to decide whether a
27 target is obsolete; they are not automatically rebuilt when they are
28 obsolete yet a target depends on them. They will only be rebuilt once the
29 dynamic call to <b>remake</b> is executed.
30 
31 In other words, the following two rules have almost the same behavior.
32 
33 @verbatim
34 target1 target2 ... : prerequisite1 prerequisite2 ...
35  shell script
36 
37 target1 target2 ... :
38  remake prerequisite1 prerequisite2 ...
39  shell script
40 @endverbatim
41 
42 (There is a difference if the targets already exist, have never been
43 built before, and the prerequisites are either younger or obsolete, since
44 the targets will not be rebuilt in the second case.)
45 
46 The above usage of dynamic dependencies is hardly useful. Their strength
47 lies in the fact that they can be computed on the fly:
48 
49 @verbatim
50 %.o : %.c
51  gcc -MMD -MF $@.d -o $@ -c $<
52  remake -r < $@.d
53  rm $@.d
54 
55 %.cmo : %.ml
56  ocamldep $< | remake -r $@
57  ocamlc -c $<
58 
59 after.xml: before.xml rules.xsl
60  xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
61  remake `sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps`
62  rm deps
63 @endverbatim
64 
65 Note that the first rule fails if any of the header files included by
66 a C source file has to be automatically generated. In that case, one
67 should perform a first call to <b>remake</b> them before calling the
68 compiler. (Dependencies from several calls to <b>remake</b> are
69 cumulative, so they will all be remembered the next time.)
70 
71 \section sec-usage Usage
72 
73 Usage: <tt>remake <i>options</i> <i>targets</i></tt>
74 
75 Options:
76 
77 - <tt>-d</tt>: Echo script commands.
78 - <tt>-f FILE</tt>: Read <tt>FILE</tt> as <b>Remakefile</b>.
79 - <tt>-j[N]</tt>, <tt>--jobs=[N]</tt>: Allow <tt>N</tt> jobs at once;
80  infinite jobs with no argument.
81 - <tt>-k</tt>, <tt>--keep-going</tt>: Keep going when some targets cannot be made.
82 - <tt>-r</tt>: Look up targets from the dependencies on standard input.
83 - <tt>-s</tt>, <tt>--silent</tt>, <tt>--quiet</tt>: Do not echo targets.
84 
85 \section sec-syntax Syntax
86 
87 Lines starting with a space character or a tabulation are assumed to be rule
88 scripts. They are only allowed after a rule header.
89 
90 Lines starting with <tt>#</tt> are considered to be comments and are ignored.
91 They do interrupt rule scripts though.
92 
93 Any other line is either a variable definition or a rule header. If such a
94 line ends with a backslash, the following line break is ignored and the line
95 extends to the next one.
96 
97 Variable definitions are a single name followed by equal followed by a list
98 of names, possibly empty.
99 
100 Rule headers are a nonempty list of names, followed by a colon, followed by
101 another list of names, possibly empty. Basically, the syntax of a rule is as
102 follows:
103 
104 @verbatim
105 targets : prerequisites
106  shell script
107 @endverbatim
108 
109 List of names are space-separated sequences of names. If a name contains a
110 space character, it should be put into double quotes. Names can not be any
111 of the following special characters <tt>:$(),="</tt>. Again, quotation
112 should be used. Quotation marks can be escaped by a backslash inside
113 quoted names.
114 
115 \subsection sec-variables Variables
116 
117 Variables can be used to factor lists of targets or prerequisites. They are
118 expanded as they are encountered during <b>Remakefile</b> parsing.
119 
120 @verbatim
121 VAR2 = a
122 VAR1 = c d
123 VAR2 += $(VAR1) b
124 $(VAR2) e :
125 @endverbatim
126 
127 Variable assignments can appear instead of prerequisites inside non-generic
128 rules with no script. They are then expanded inside the corresponding
129 generic rule.
130 
131 @verbatim
132 foo.o: CFLAGS += -DBAR
133 
134 %.o : %.c
135  gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
136  remake -r < $@.d
137  rm $@.d
138 @endverbatim
139 
140 Note: contrarily to <b>make</b>, variable names have to be enclosed in
141 parentheses. For instance, <tt>$y</tt> is not a shorthand for <tt>\$(y)</tt> and
142 is left unexpanded.
143 
144 \subsection sec-autovars Automatic variables
145 
146 The following special symbols can appear inside scripts:
147 
148 - <tt>$&lt;</tt> expands to the first static prerequisite of the rule.
149 - <tt>$^</tt> expands to all the static prerequisites of the rule, including
150  duplicates if any.
151 - <tt>$\@</tt> expands to the first target of the rule.
152 - <tt>$*</tt> expands to the string that matched <tt>%</tt> in a generic rule.
153 - <tt>$$</tt> expands to a single dollar symbol.
154 
155 Note: contrarily to <b>make</b>, there are no corresponding variables. For
156 instance, <tt>$^</tt> is not a shorthand for <tt>$(^)</tt>. Another
157 difference is that <tt>$\@</tt> is always the first target, not the one that
158 triggered the rule.
159 
160 \subsection sec-functions Built-in functions
161 
162 <b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
163 
164 - <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
165  by prepending its first argument to each element of its second argument.
166 - <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
167  by appending its first argument to each element of its second argument.
168 
169 \subsection sec-order Order-only prerequisites
170 
171 If the static prerequisites of a rule contain a pipe symbol, prerequisites
172 on its right do not cause the targets to become obsolete if they are newer
173 (unless they are also dynamically registered as dependencies). They are
174 meant to be used when the targets do not directly depend on them, but the
175 computation of their dynamic dependencies does.
176 
177 @verbatim
178 %.o : %.c | parser.h
179  gcc -MMD -MF $@.d -o $@ -c $<
180  remake -r < $@.d
181  rm $@.d
182 
183 parser.c parser.h: parser.y
184  yacc -d -o parser.c parser.y
185 @endverbatim
186 
187 \subsection sec-special-tgt Special targets
188 
189 Target <tt>.PHONY</tt> marks its prerequisites as being always obsolete.
190 
191 \subsection sec-special-var Special variables
192 
193 Variable <tt>.OPTIONS</tt> is handled specially. Its content enables some
194 features of <b>remake</b> that are not enabled by default.
195 
196 - <tt>variable-propagation</tt>: When a variable is set in the prerequisite
197  part of a rule, it is propagated to the rules of all the targets this rule
198  depends on. This option also enables variables to be set on the command
199  line. Note that, as in <b>make</b>, this features introduces non-determinism:
200  the content of some variables will depend on the build order.
201 
202 \section sec-semantics Semantics
203 
204 \subsection src-obsolete When are targets obsolete?
205 
206 A target is obsolete:
207 
208 - if there is no file corresponding to the target, or to one of its siblings
209  in a multi-target rule,
210 - if any of its dynamic prerequisites from a previous run or any of its static
211  prerequisites is obsolete,
212 - if the latest file corresponding to its siblings or itself is older than any
213  of its dynamic prerequisites or static prerequisites.
214 
215 In all the other cases, it is assumed to be up-to-date (and so are all its
216 siblings). Note that the last rule above says "latest" and not "earliest". While
217 it might cause some obsolete targets to go unnoticed in corner cases, it allows
218 for the following kind of rules:
219 
220 @verbatim
221 config.h stamp-config_h: config.h.in config.status
222  ./config.status config.h
223  touch stamp-config_h
224 @endverbatim
225 
226 A <tt>config.status</tt> file generally does not update header files (here
227 <tt>config.h</tt>) if they would not change. As a consequence, if not for the
228 <tt>stamp-config_h</tt> file above, a header would always be considered obsolete
229 once one of its prerequisites is modified. Note that touching <tt>config.h</tt>
230 rather than <tt>stamp-config_h</tt> would defeat the point of not updating it
231 in the first place, since the program files would need to be rebuilt.
232 
233 Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
234 checks whether the target still needs to be built. If it was obsolete only
235 because its prerequisites needed to be rebuilt and none of them changed, the
236 target is assumed to be up-to-date.
237 
238 \subsection sec-rules How are targets (re)built?
239 
240 There are two kinds of rules. If any of the targets or prerequisites contains
241 a <tt>%</tt> character, the rule is said to be <em>generic</em>. All the
242 targets of the rule shall then contain a single <tt>%</tt> character. All the
243 other rules are said to be <em>specific</em>.
244 
245 A rule is said to <em>match</em> a given target:
246 
247 - if it is specific and the target appears inside its target list,
248 - if it is generic and there is a way to replace the <tt>%</tt> character
249  from one of its targets so that it matches the given target.
250 
251 When <b>remake</b> tries to build a given target, it looks for a specific rule
252 that matches it. If there is one and its script is nonempty, it uses it to
253 rebuild the target.
254 
255 Otherwise, it looks for a generic rule that matches the target. If there are
256 several matching rules, it chooses the one with the shortest pattern (and if
257 there are several ones, the earliest one). <b>remake</b> then looks for
258 specific rules that match each target of the generic rule. All the
259 prerequisites of these specific rules are added to those of the generic rule.
260 The script of the generic rule is used to build the target.
261 
262 Example:
263 
264 @verbatim
265 t%1 t2%: p1 p%2
266  commands building t%1 and t2%
267 
268 t2z: p4
269  commands building t2z
270 
271 ty1: p3
272 
273 # t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
274 # t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
275 # t2z is built by the second rule and its prerequisite is p4
276 @endverbatim
277 
278 The set of rules from <b>Remakefile</b> is ill-formed:
279 
280 - if any specific rule matching a target of the generic rule has a nonempty script,
281 - if any target of the generic rule is matched by a generic rule with a shorter pattern.
282 
283 \section sec-compilation Compilation
284 
285 - On Linux, MacOSX, and BSD: <tt>g++ -o remake remake.cpp</tt>
286 - On Windows: <tt>g++ -o remake.exe remake.cpp -lws2_32</tt>
287 
288 Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
289 specify the path to the executable for its recursive calls. Thanks to its
290 single source file, <b>remake</b> can be shipped inside other packages and
291 built at configuration time.
292 
293 \section sec-differences Differences with other build systems
294 
295 Differences with <b>make</b>:
296 
297 - Dynamic dependencies are supported.
298 - For rules with multiple targets, the shell script is executed only once
299  and is assumed to build all the targets. There is no need for
300  convoluted rules that are robust enough for parallel builds. For generic
301  rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
302 - As with <b>redo</b>, only one shell is run when executing a script,
303  rather than one per script line. Note that the shells are run with
304  option <tt>-e</tt>, thus causing them to exit as soon as an error is
305  encountered.
306 - The prerequisites of generic rules (known as implicit rules in make lingo)
307  are not used to decide between several of them. <b>remake</b> does not
308  select one for which it could satisfy the dependencies.
309 - Variables and built-in functions are expanded as they are encountered
310  during <b>Remakefile</b> parsing.
311 - Target-specific variables are not propagated, unless specifically enabled,
312  since this causes non-deterministic builds. This is the same for variables
313  set on the command line.
314 
315 Differences with <b>redo</b>:
316 
317 - As with <b>make</b>, it is possible to write the following kind of rules
318  in <b>remake</b>.
319 @verbatim
320 Remakefile: Remakefile.in ./config.status
321  ./config.status Remakefile
322 @endverbatim
323 - If a target is already built the first time <b>remake</b> runs, it still
324  uses the static prerequisites of rules mentioning it to check whether it
325  needs to be rebuilt. It does not assume it to be up-to-date. As with
326  <b>redo</b> though, if its obsolete status would be due to a dynamic
327  prerequisite, it will go unnoticed; it should be removed beforehand.
328 - Multiple targets are supported.
329 - <b>remake</b> has almost no features: no checksum-based dependencies, no
330  compatibility with job servers, etc.
331 
332 \section sec-limitations Limitations
333 
334 - If a rule script calls <b>remake</b>, the current working directory should
335  be the directory containing <b>Remakefile</b> (or the working directory
336  from the original <b>remake</b> if it was called with option <b>-f</b>).
337 - As with <b>make</b>, variables passed on the command line should keep
338  the same values, to ensure deterministic builds.
339 - Some cases of ill-formed rules are not caught by <b>remake</b> and can
340  thus lead to unpredictable behaviors.
341 
342 \section sec-links Links
343 
344 @see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
345 https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
346 
347 \section sec-licensing Licensing
348 
349 @author Guillaume Melquiond
350 @version 0.11
351 @date 2012-2013
352 @copyright
353 This program is free software: you can redistribute it and/or modify
354 it under the terms of the GNU General Public License as published by
355 the Free Software Foundation, either version 3 of the License, or
356 (at your option) any later version.
357 \n
358 This program is distributed in the hope that it will be useful,
359 but WITHOUT ANY WARRANTY; without even the implied warranty of
360 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
361 GNU General Public License for more details.
362 
363 \section sec-internals Internals
364 
365 The parent <b>remake</b> process acts as a server. The other ones have a
366 REMAKE_SOCKET environment variable that tells them how to contact the
367 server. They send the content of the REMAKE_JOB_ID environment variable,
368 so that the server can associate the child targets to the jobs that
369 spawned them. They then wait for completion and exit with the status
370 returned by the server. This is handled by #client_mode.
371 
372 The server calls #load_dependencies and #save_dependencies to serialize
373 dynamic dependencies from <b>.remake</b>. It loads <b>Remakefile</b> with
374 #load_rules. It then runs #server_mode, which calls #server_loop.
375 
376 When building a target, the following sequence of events happens:
377 
378 - #start calls #find_rule (and #find_generic_rule) to get the rule.
379 - It then creates a pseudo-client if the rule has static dependencies, or
380  calls #run_script otherwise. In both cases, a new job is created; the
381  rule and the variables are stored into #jobs.
382 - #run_script creates a shell process and stores it in #job_pids. It
383  increases #running_jobs.
384 - The child process possibly calls <b>remake</b> with a list of targets.
385 - #accept_client receives a build request from a child process and adds
386  it to #clients. It also records the new dependencies of the job into
387  #dependencies. It increases #waiting_jobs.
388 - #handle_clients uses #get_status to look up the obsoleteness of the
389  targets.
390 - Once the targets of a request have been built or one of them has failed,
391  #handle_clients calls #complete_request and removes the request from
392  #clients.
393 - If the build targets come from a pseudo-client, #complete_request calls
394  #run_script. Otherwise it sends the reply to the corresponding child
395  process and decreases #waiting_jobs.
396 - When a child process ends, #server_loop calls #finalize_job, which
397  removes the process from #job_pids, decreases #running_jobs, and calls
398  #complete_job.
399 - #complete_job removes the job from #jobs and calls #update_status
400  to change the status of the targets. It also removes the target files in
401  case of failure.
402 */
403 
404 #ifdef _WIN32
405 #define WIN32_LEAN_AND_MEAN
406 #define WINDOWS
407 #endif
408 
409 #include <fstream>
410 #include <iostream>
411 #include <list>
412 #include <map>
413 #include <set>
414 #include <sstream>
415 #include <string>
416 #include <vector>
417 #include <cassert>
418 #include <cstdlib>
419 #include <cstring>
420 #include <ctime>
421 #include <errno.h>
422 #include <fcntl.h>
423 #include <signal.h>
424 #include <unistd.h>
425 #include <sys/stat.h>
426 #include <sys/types.h>
427 
428 #ifdef __APPLE__
429 #define MACOSX
430 #endif
431 
432 #ifdef __linux__
433 #define LINUX
434 #endif
435 
436 #ifdef WINDOWS
437 #include <windows.h>
438 #include <winbase.h>
439 #include <winsock2.h>
440 #define pid_t HANDLE
441 typedef SOCKET socket_t;
442 #else
443 #include <sys/socket.h>
444 #include <sys/un.h>
445 #include <sys/wait.h>
446 typedef int socket_t;
447 enum { INVALID_SOCKET = -1 };
448 extern char **environ;
449 #endif
450 
451 #if defined(WINDOWS) || defined(MACOSX)
452 enum { MSG_NOSIGNAL = 0 };
453 #endif
454 
455 typedef std::list<std::string> string_list;
456 
457 typedef std::set<std::string> string_set;
458 
459 /**
460  * Reference-counted shared object.
461  * @note The default constructor delays the creation of the object until it
462  * is first dereferenced.
463  */
464 template<class T>
465 struct ref_ptr
466 {
467  struct content
468  {
469  size_t cnt;
470  T val;
471  content(): cnt(1) {}
472  content(T const &t): cnt(1), val(t) {}
473  };
474  mutable content *ptr;
475  ref_ptr(): ptr(NULL) {}
476  ref_ptr(T const &t): ptr(new content(t)) {}
477  ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
478  ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
480  {
481  if (ptr == p.ptr) return *this;
482  if (ptr && --ptr->cnt == 0) delete ptr;
483  ptr = p.ptr;
484  if (ptr) ++ptr->cnt;
485  return *this;
486  }
487  T &operator*() const
488  {
489  if (!ptr) ptr = new content;
490  return ptr->val;
491  }
492  T *operator->() const { return &**this; }
493 };
494 
496 {
499 };
500 
501 typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
502 
503 typedef std::map<std::string, string_list> variable_map;
504 
505 /**
506  * Build status of a target.
507  */
509 {
510  Uptodate, ///< Target is up-to-date.
511  Todo, ///< Target is missing or obsolete.
512  Recheck, ///< Target has an obsolete dependency.
513  Running, ///< Target is being rebuilt.
514  Remade, ///< Target was successfully rebuilt.
515  Failed ///< Build failed for target.
516 };
517 
518 /**
519  * Build status of a target.
520  */
521 struct status_t
522 {
523  status_e status; ///< Actual status.
524  time_t last; ///< Last-modified date.
525 };
526 
527 typedef std::map<std::string, status_t> status_map;
528 
529 /**
530  * Delayed assignment to a variable.
531  */
532 struct assign_t
533 {
534  bool append;
536 };
537 
538 typedef std::map<std::string, assign_t> assign_map;
539 
540 /**
541  * A rule loaded from Remakefile.
542  */
543 struct rule_t
544 {
545  string_list targets; ///< Files produced by this rule.
546  string_list deps; ///< Dependencies used for an implicit call to remake at the start of the script.
547  string_list wdeps; ///< Like #deps, except that they are not registered as dependencies.
548  assign_map assigns; ///< Assignment of variables.
549  std::string script; ///< Shell script for building the targets.
550 };
551 
552 typedef std::list<rule_t> rule_list;
553 
554 typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
555 
556 /**
557  * A job created from a set of rules.
558  */
559 
560 struct job_t
561 {
562  rule_t rule; ///< Original rule.
563  std::string stem; ///< Pattern used to instantiate the generic rule, if any.
564  variable_map vars; ///< Values of local variables.
565 };
566 
567 typedef std::map<int, job_t> job_map;
568 
569 typedef std::map<pid_t, int> pid_job_map;
570 
571 /**
572  * Client waiting for a request to complete.
573  *
574  * There are two kinds of clients:
575  * - real clients, which are instances of remake created by built scripts,
576  * - pseudo clients, which are created by the server to build specific targets.
577  *
578  * Among pseudo clients, there are two categories:
579  * - original clients, which are created for the targets passed on the
580  * command line by the user or for the initial regeneration of the rule file,
581  * - dependency clients, which are created to handle rules that have
582  * explicit dependencies and thus to emulate a call to remake.
583  */
584 struct client_t
585 {
586  socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
587  int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
588  bool failed; ///< Whether some targets failed in mode -k.
589  string_list pending; ///< Targets not yet started.
590  string_set running; ///< Targets being built.
591  variable_map vars; ///< Variables set on request.
592  bool delayed; ///< Whether it is a dependency client and a script has to be started on request completion.
593  client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(false) {}
594 };
595 
596 typedef std::list<client_t> client_list;
597 
598 /**
599  * Map from variable names to their content.
600  * Initialized with the values passed on the command line.
601  */
603 
604 /**
605  * Map from targets to their known dependencies.
606  */
608 
609 /**
610  * Map from targets to their build status.
611  */
613 
614 /**
615  * Set of generic rules loaded from Remakefile.
616  */
618 
619 /**
620  * Map from targets to specific rules loaded from Remakefile.
621  */
623 
624 /**
625  * Map of jobs being built.
626  */
627 static job_map jobs;
628 
629 /**
630  * Map from jobs to shell pids.
631  */
633 
634 /**
635  * List of clients waiting for a request to complete.
636  * New clients are put to front, so that the build process is depth-first.
637  */
639 
640 /**
641  * Maximum number of parallel jobs (non-positive if unbounded).
642  * Can be modified by the -j option.
643  */
644 static int max_active_jobs = 1;
645 
646 /**
647  * Whether to keep building targets in case of failure.
648  * Can be modified by the -k option.
649  */
650 static bool keep_going = false;
651 
652 /**
653  * Number of jobs currently running:
654  * - it increases when a process is created in #run_script,
655  * - it decreases when a completion message is received in #finalize_job.
656  *
657  * @note There might be some jobs running while #clients is empty.
658  * Indeed, if a client requested two targets to be rebuilt, if they
659  * are running concurrently, if one of them fails, the client will
660  * get a failure notice and might terminate before the other target
661  * finishes.
662  */
663 static int running_jobs = 0;
664 
665 /**
666  * Number of jobs currently waiting for a build request to finish:
667  * - it increases when a build request is received in #accept_client
668  * (since the client is presumably waiting for the reply),
669  * - it decreases when a reply is sent in #complete_request.
670  */
671 static int waiting_jobs = 0;
672 
673 /**
674  * Global counter used to produce increasing job numbers.
675  * @see jobs
676  */
677 static int job_counter = 0;
678 
679 /**
680  * Socket on which the server listens for client request.
681  */
683 
684 /**
685  * Whether the request of an original client failed.
686  */
687 static bool build_failure;
688 
689 #ifndef WINDOWS
690 /**
691  * Name of the server socket in the file system.
692  */
693 static char *socket_name;
694 #endif
695 
696 /**
697  * Name of the first target of the first specific rule, used for default run.
698  */
699 static std::string first_target;
700 
701 /**
702  * Whether a short message should be displayed for each target.
703  */
704 static bool show_targets = true;
705 
706 /**
707  * Whether script commands are echoed.
708  */
709 static bool echo_scripts = false;
710 
711 /**
712  * Time at the start of the program.
713  */
714 static time_t now = time(NULL);
715 
716 /**
717  * Directory with respect to which command-line names are relative.
718  */
719 static std::string working_dir;
720 
721 /**
722  * Directory with respect to which targets are relative.
723  */
724 static std::string prefix_dir;
725 
726 /**
727  * Whether the prefix directory is different from #working_dir.
728  */
729 static bool changed_prefix_dir;
730 
731 /**
732  * Whether target-specific variables are propagated to prerequisites.
733  */
734 static bool propagate_vars = false;
735 
736 #ifndef WINDOWS
737 static volatile sig_atomic_t got_SIGCHLD = 0;
738 
739 static void sigchld_handler(int)
740 {
741  got_SIGCHLD = 1;
742 }
743 
744 static void sigint_handler(int)
745 {
746  // Child processes will receive the signal too, so just prevent
747  // new jobs from starting and wait for the running jobs to fail.
748  keep_going = false;
749 }
750 #endif
751 
752 struct log
753 {
754  bool active, open;
755  int depth;
756  log(): active(false), open(false), depth(0)
757  {
758  }
759  std::ostream &operator()()
760  {
761  if (open) std::cerr << std::endl;
762  assert(depth >= 0);
763  std::cerr << std::string(depth * 2, ' ');
764  open = false;
765  return std::cerr;
766  }
767  std::ostream &operator()(bool o)
768  {
769  if (o && open) std::cerr << std::endl;
770  if (!o) --depth;
771  assert(depth >= 0);
772  if (o || !open) std::cerr << std::string(depth * 2, ' ');
773  if (o) ++depth;
774  open = o;
775  return std::cerr;
776  }
777 };
778 
780 
782 {
785  {
786  }
788  {
789  if (debug.active && still_open) debug(false) << "done\n";
790  }
791 };
792 
793 #define DEBUG if (debug.active) debug()
794 #define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
795 #define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
796 
797 /**
798  * Strong typedef for strings that need escaping.
799  * @note The string is stored as a reference, so the constructed object is
800  * meant to be immediately consumed.
801  */
803 {
804  std::string const &input;
805  escape_string(std::string const &s): input(s) {}
806 };
807 
808 /**
809  * Write the string in @a se to @a out if it does not contain any special
810  * characters, a quoted and escaped string otherwise.
811  */
812 static std::ostream &operator<<(std::ostream &out, escape_string const &se)
813 {
814  std::string const &s = se.input;
815  char const *quoted_char = ",: '";
816  char const *escaped_char = "\"\\$!";
817  bool need_quotes = false;
818  char *buf = NULL;
819  size_t len = s.length(), last = 0, j = 0;
820  for (size_t i = 0; i < len; ++i)
821  {
822  if (strchr(escaped_char, s[i]))
823  {
824  need_quotes = true;
825  if (!buf) buf = new char[len * 2];
826  memcpy(&buf[j], &s[last], i - last);
827  j += i - last;
828  buf[j++] = '\\';
829  buf[j++] = s[i];
830  last = i + 1;
831  }
832  if (!need_quotes && strchr(quoted_char, s[i]))
833  need_quotes = true;
834  }
835  if (!need_quotes) return out << s;
836  out << '"';
837  if (!buf) return out << s << '"';
838  out.write(buf, j);
839  out.write(&s[last], len - last);
840  delete[] buf;
841  return out << '"';
842 }
843 
844 /**
845  * @defgroup paths Path helpers
846  *
847  * @{
848  */
849 
850 /**
851  * Initialize #working_dir.
852  */
853 static void init_working_dir()
854 {
855  char buf[1024];
856  char *res = getcwd(buf, sizeof(buf));
857  if (!res)
858  {
859  perror("Failed to get working directory");
860  exit(EXIT_FAILURE);
861  }
862  working_dir = buf;
863 #ifdef WINDOWS
864  for (size_t i = 0, l = working_dir.size(); i != l; ++i)
865  {
866  if (working_dir[i] == '\\') working_dir[i] = '/';
867  }
868 #endif
870 }
871 
872 /**
873  * Initialize #prefix_dir and switch to it.
874  */
875 static void init_prefix_dir()
876 {
877  for (;;)
878  {
879  struct stat s;
880  if (stat((prefix_dir + "/Remakefile").c_str(), &s) == 0)
881  {
882  if (!changed_prefix_dir) return;
883  if (chdir(prefix_dir.c_str()))
884  {
885  perror("Failed to change working directory");
886  exit(EXIT_FAILURE);
887  }
888  if (show_targets)
889  {
890  std::cout << "remake: Entering directory `" << prefix_dir << '\'' << std::endl;
891  }
892  return;
893  }
894  size_t pos = prefix_dir.find_last_of('/');
895  if (pos == std::string::npos)
896  {
897  std::cerr << "Failed to locate Remakefile in the current directory or one of its parents" << std::endl;
898  exit(EXIT_FAILURE);
899  }
900  prefix_dir.erase(pos);
901  changed_prefix_dir = true;
902  }
903 }
904 
905 /**
906  * Normalize an absolute path with respect to @a p.
907  * Paths outside the subtree are left unchanged.
908  */
909 static std::string normalize_abs(std::string const &s, std::string const &p)
910 {
911  size_t l = p.length();
912  if (s.compare(0, l, p)) return s;
913  size_t ll = s.length();
914  if (ll == l) return ".";
915  if (s[l] != '/')
916  {
917  size_t pos = s.rfind('/', l);
918  assert(pos != std::string::npos);
919  return s.substr(pos + 1);
920  }
921  if (ll == l + 1) return ".";
922  return s.substr(l + 1);
923 }
924 
925 /**
926  * Normalize path @a s (possibly relative to @a w) with respect to @a p.
927  *
928  * - If both @a p and @a w are empty, the function just removes ".", "..", "//".
929  * - If only @a p is empty, the function returns an absolute path.
930  */
931 static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
932 {
933 #ifdef WINDOWS
934  char const *delim = "/\\";
935 #else
936  char delim = '/';
937 #endif
938  size_t pos = s.find_first_of(delim);
939  if (pos == std::string::npos && w == p) return s;
940  bool absolute = pos == 0;
941  if (!absolute && w != p && !w.empty())
942  return normalize(w + '/' + s, w, p);
943  size_t prev = 0, len = s.length();
944  string_list l;
945  for (;;)
946  {
947  if (pos != prev)
948  {
949  std::string n = s.substr(prev, pos - prev);
950  if (n == "..")
951  {
952  if (!l.empty()) l.pop_back();
953  else if (!absolute && !w.empty())
954  return normalize(w + '/' + s, w, p);
955  }
956  else if (n != ".")
957  l.push_back(n);
958  }
959  ++pos;
960  if (pos >= len) break;
961  prev = pos;
962  pos = s.find_first_of(delim, prev);
963  if (pos == std::string::npos) pos = len;
964  }
965  string_list::const_iterator i = l.begin(), i_end = l.end();
966  if (i == i_end) return absolute ? "/" : ".";
967  std::string n;
968  if (absolute) n.push_back('/');
969  n.append(*i);
970  for (++i; i != i_end; ++i)
971  {
972  n.push_back('/');
973  n.append(*i);
974  }
975  if (absolute && !p.empty()) return normalize_abs(n, p);
976  return n;
977 }
978 
979 /**
980  * Normalize the content of a list of targets.
981  */
982 static void normalize_list(string_list &l, std::string const &w, std::string const &p)
983 {
984  for (string_list::iterator i = l.begin(),
985  i_end = l.end(); i != i_end; ++i)
986  {
987  *i = normalize(*i, w, p);
988  }
989 }
990 
991 /** @} */
992 
993 /**
994  * @defgroup lexer Lexer
995  *
996  * @{
997  */
998 
999 /**
1000  * Skip spaces.
1001  */
1002 static void skip_spaces(std::istream &in)
1003 {
1004  char c;
1005  while (strchr(" \t", (c = in.get()))) {}
1006  if (in.good()) in.putback(c);
1007 }
1008 
1009 /**
1010  * Skip empty lines.
1011  */
1012 static void skip_empty(std::istream &in)
1013 {
1014  char c;
1015  while (strchr("\r\n", (c = in.get()))) {}
1016  if (in.good()) in.putback(c);
1017 }
1018 
1019 /**
1020  * Skip end of line. If @a multi is true, skip the following empty lines too.
1021  * @return true if there was a line to end.
1022  */
1023 static bool skip_eol(std::istream &in, bool multi = false)
1024 {
1025  char c = in.get();
1026  if (c == '\r') c = in.get();
1027  if (c != '\n' && in.good()) in.putback(c);
1028  if (c != '\n' && !in.eof()) return false;
1029  if (multi) skip_empty(in);
1030  return true;
1031 }
1032 
1033 enum
1034 {
1036  Word = 1 << 1,
1037  Colon = 1 << 2,
1038  Equal = 1 << 3,
1039  Dollarpar = 1 << 4,
1040  Rightpar = 1 << 5,
1041  Comma = 1 << 6,
1042  Plusequal = 1 << 7,
1043  Pipe = 1 << 8,
1044 };
1045 
1046 /**
1047  * Skip spaces and peek at the next token.
1048  * If it is one of @a mask, skip it (if it is not Word) and return it.
1049  * @note For composite tokens allowed by @a mask, input characters might
1050  * have been eaten even for an Unexpected result.
1051  */
1052 static int expect_token(std::istream &in, int mask)
1053 {
1054  while (true)
1055  {
1056  skip_spaces(in);
1057  char c = in.peek();
1058  if (!in.good()) return Unexpected;
1059  int tok;
1060  switch (c)
1061  {
1062  case '\r':
1063  case '\n': return Unexpected;
1064  case ':': tok = Colon; break;
1065  case ',': tok = Comma; break;
1066  case '=': tok = Equal; break;
1067  case ')': tok = Rightpar; break;
1068  case '|': tok = Pipe; break;
1069  case '$':
1070  if (!(mask & Dollarpar)) return Unexpected;
1071  in.ignore(1);
1072  tok = Dollarpar;
1073  if (in.peek() != '(') return Unexpected;
1074  break;
1075  case '+':
1076  if (!(mask & Plusequal)) return Unexpected;
1077  in.ignore(1);
1078  tok = Plusequal;
1079  if (in.peek() != '=') return Unexpected;
1080  break;
1081  case '\\':
1082  in.ignore(1);
1083  if (skip_eol(in)) continue;
1084  in.putback('\\');
1085  return mask & Word ? Word : Unexpected;
1086  default:
1087  return mask & Word ? Word : Unexpected;
1088  }
1089  if (!(tok & mask)) return Unexpected;
1090  in.ignore(1);
1091  return tok;
1092  }
1093 }
1094 
1095 /**
1096  * Read a (possibly quoted) word.
1097  */
1098 static std::string read_word(std::istream &in, bool detect_equal = true)
1099 {
1100  int c = in.peek();
1101  std::string res;
1102  if (!in.good()) return res;
1103  char const *separators = " \t\r\n$(),:";
1104  bool quoted = c == '"';
1105  if (quoted) in.ignore(1);
1106  bool plus = false;
1107  while (true)
1108  {
1109  c = in.peek();
1110  if (!in.good()) return res;
1111  if (quoted)
1112  {
1113  in.ignore(1);
1114  if (c == '\\')
1115  res += in.get();
1116  else if (c == '"')
1117  quoted = false;
1118  else
1119  res += c;
1120  continue;
1121  }
1122  if (detect_equal && c == '=')
1123  {
1124  if (plus) in.putback('+');
1125  return res;
1126  }
1127  if (plus)
1128  {
1129  res += '+';
1130  plus = false;
1131  }
1132  if (strchr(separators, c)) return res;
1133  in.ignore(1);
1134  if (detect_equal && c == '+') plus = true;
1135  else res += c;
1136  }
1137 }
1138 
1139 /** @} */
1140 
1141 /**
1142  * @defgroup stream Token streams
1143  *
1144  * @{
1145  */
1146 
1147 /**
1148  * Possible results from word producers.
1149  */
1151 {
1155 };
1156 
1157 /**
1158  * Interface for word producers.
1159  */
1161 {
1162  virtual ~generator() {}
1163  virtual input_status next(std::string &) = 0;
1164 };
1165 
1166 /**
1167  * Generator for the words of a variable.
1168  */
1170 {
1171  std::string name;
1172  string_list::const_iterator vcur, vend;
1173  variable_generator(std::string const &, variable_map const *);
1174  input_status next(std::string &);
1175 };
1176 
1178  variable_map const *local_variables): name(n)
1179 {
1180  if (local_variables)
1181  {
1182  variable_map::const_iterator i = local_variables->find(name);
1183  if (i != local_variables->end())
1184  {
1185  vcur = i->second.begin();
1186  vend = i->second.end();
1187  return;
1188  }
1189  }
1190  variable_map::const_iterator i = variables.find(name);
1191  if (i == variables.end()) return;
1192  vcur = i->second.begin();
1193  vend = i->second.end();
1194 }
1195 
1197 {
1198  if (vcur != vend)
1199  {
1200  res = *vcur;
1201  ++vcur;
1202  return Success;
1203  }
1204  return Eof;
1205 }
1206 
1207 /**
1208  * Generator for the words of an input stream.
1209  */
1211 {
1212  std::istream &in;
1216  input_generator(std::istream &i, variable_map const *lv, bool e = false)
1217  : in(i), nested(NULL), local_variables(lv), earliest_exit(e), done(false) {}
1218  input_status next(std::string &);
1219  ~input_generator() { assert(!nested); }
1220 };
1221 
1222 static generator *get_function(input_generator const &, std::string const &);
1223 
1225 {
1226  if (nested)
1227  {
1228  restart:
1229  input_status s = nested->next(res);
1230  if (s == Success) return Success;
1231  delete nested;
1232  nested = NULL;
1233  if (s == SyntaxError) return SyntaxError;
1234  }
1235  if (done) return Eof;
1236  if (earliest_exit) done = true;
1237  switch (expect_token(in, Word | Dollarpar))
1238  {
1239  case Word:
1240  res = read_word(in, false);
1241  return Success;
1242  case Dollarpar:
1243  {
1244  std::string name = read_word(in, false);
1245  if (name.empty()) return SyntaxError;
1246  if (expect_token(in, Rightpar))
1248  else
1249  {
1250  nested = get_function(*this, name);
1251  if (!nested) return SyntaxError;
1252  }
1253  goto restart;
1254  }
1255  default:
1256  return Eof;
1257  }
1258 }
1259 
1260 /**
1261  * Read a list of words from an input generator.
1262  * @return false if a syntax error was encountered.
1263  */
1265 {
1266  while (true)
1267  {
1268  res.push_back(std::string());
1269  input_status s = in.next(res.back());
1270  if (s == Success) continue;
1271  res.pop_back();
1272  return s == Eof;
1273  }
1274 }
1275 
1276 static bool read_words(std::istream &in, string_list &res)
1277 {
1278  input_generator gen(in, NULL);
1279  return read_words(gen, res);
1280 }
1281 
1282 /**
1283  * Generator for the result of function addprefix.
1284  */
1286 {
1289  string_list::const_iterator prei;
1290  size_t prej, prel;
1291  std::string suf;
1292  addprefix_generator(input_generator const &, bool &);
1293  input_status next(std::string &);
1294 };
1295 
1297  : gen(top.in, top.local_variables)
1298 {
1299  if (!read_words(gen, pre)) return;
1300  if (!expect_token(gen.in, Comma)) return;
1301  prej = 0;
1302  prel = pre.size();
1303  ok = true;
1304 }
1305 
1307 {
1308  if (prej)
1309  {
1310  produce:
1311  if (prej == prel)
1312  {
1313  res = *prei + suf;
1314  prej = 0;
1315  }
1316  else
1317  {
1318  res = *prei++;
1319  ++prej;
1320  }
1321  return Success;
1322  }
1323  switch (gen.next(res))
1324  {
1325  case Success:
1326  if (!prel) return Success;
1327  prei = pre.begin();
1328  prej = 1;
1329  suf = res;
1330  goto produce;
1331  case Eof:
1332  return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
1333  default:
1334  return SyntaxError;
1335  }
1336 }
1337 
1338 /**
1339  * Generator for the result of function addsuffix.
1340  */
1342 {
1345  string_list::const_iterator sufi;
1346  size_t sufj, sufl;
1347  std::string pre;
1348  addsuffix_generator(input_generator const &, bool &);
1349  input_status next(std::string &);
1350 };
1351 
1353  : gen(top.in, top.local_variables)
1354 {
1355  if (!read_words(gen, suf)) return;
1356  if (!expect_token(gen.in, Comma)) return;
1357  sufj = 0;
1358  sufl = suf.size();
1359  ok = true;
1360 }
1361 
1363 {
1364  if (sufj)
1365  {
1366  if (sufj != sufl)
1367  {
1368  res = *sufi++;
1369  ++sufj;
1370  return Success;
1371  }
1372  sufj = 0;
1373  }
1374  switch (gen.next(res))
1375  {
1376  case Success:
1377  if (!sufl) return Success;
1378  sufi = suf.begin();
1379  sufj = 1;
1380  res += *sufi++;
1381  return Success;
1382  case Eof:
1383  return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
1384  default:
1385  return SyntaxError;
1386  }
1387 }
1388 
1389 /**
1390  * Return a generator for function @a name.
1391  */
1392 static generator *get_function(input_generator const &in, std::string const &name)
1393 {
1394  skip_spaces(in.in);
1395  generator *g = NULL;
1396  bool ok = false;
1397  if (name == "addprefix") g = new addprefix_generator(in, ok);
1398  else if (name == "addsuffix") g = new addsuffix_generator(in, ok);
1399  if (!g || ok) return g;
1400  delete g;
1401  return NULL;
1402 }
1403 
1404 /** @} */
1405 
1406 /**
1407  * @defgroup database Dependency database
1408  *
1409  * @{
1410  */
1411 
1412 /**
1413  * Load dependencies from @a in.
1414  */
1415 static void load_dependencies(std::istream &in)
1416 {
1417  if (false)
1418  {
1419  error:
1420  std::cerr << "Failed to load database" << std::endl;
1421  exit(EXIT_FAILURE);
1422  }
1423 
1424  while (!in.eof())
1425  {
1426  string_list targets;
1427  if (!read_words(in, targets)) goto error;
1428  if (in.eof()) return;
1429  if (targets.empty()) goto error;
1430  DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1431  if (in.get() != ':') goto error;
1433  dep->targets = targets;
1434  string_list deps;
1435  if (!read_words(in, deps)) goto error;
1436  dep->deps.insert(deps.begin(), deps.end());
1437  for (string_list::const_iterator i = targets.begin(),
1438  i_end = targets.end(); i != i_end; ++i)
1439  {
1440  dependencies[*i] = dep;
1441  }
1442  skip_empty(in);
1443  }
1444 }
1445 
1446 /**
1447  * Load known dependencies from file <tt>.remake</tt>.
1448  */
1449 static void load_dependencies()
1450 {
1451  DEBUG_open << "Loading database... ";
1452  std::ifstream in(".remake");
1453  if (!in.good())
1454  {
1455  DEBUG_close << "not found\n";
1456  return;
1457  }
1458  load_dependencies(in);
1459 }
1460 
1461 
1462 /**
1463  * Save all the dependencies in file <tt>.remake</tt>.
1464  */
1465 static void save_dependencies()
1466 {
1467  DEBUG_open << "Saving database... ";
1468  std::ofstream db(".remake");
1469  while (!dependencies.empty())
1470  {
1471  ref_ptr<dependency_t> dep = dependencies.begin()->second;
1472  for (string_list::const_iterator i = dep->targets.begin(),
1473  i_end = dep->targets.end(); i != i_end; ++i)
1474  {
1475  db << escape_string(*i) << ' ';
1476  dependencies.erase(*i);
1477  }
1478  db << ':';
1479  for (string_set::const_iterator i = dep->deps.begin(),
1480  i_end = dep->deps.end(); i != i_end; ++i)
1481  {
1482  db << ' ' << escape_string(*i);
1483  }
1484  db << std::endl;
1485  }
1486 }
1487 
1488 /** @} */
1489 
1490 static void merge_rule(rule_t &dest, rule_t const &src);
1491 
1492 /**
1493  * @defgroup parser Rule parser
1494  *
1495  * @{
1496  */
1497 
1498 /**
1499  * Register a specific rule with an empty script:
1500  *
1501  * - Check that none of the targets already has an associated rule with a
1502  * nonempty script.
1503  * - Create a new rule with a single target for each target, if needed.
1504  * - Add the prerequisites of @a rule to all these associated rules.
1505  */
1506 static void register_transparent_rule(rule_t const &rule, string_list const &targets)
1507 {
1508  assert(rule.script.empty());
1509  for (string_list::const_iterator i = targets.begin(),
1510  i_end = targets.end(); i != i_end; ++i)
1511  {
1512  std::pair<rule_map::iterator, bool> j =
1513  specific_rules.insert(std::make_pair(*i, ref_ptr<rule_t>()));
1514  ref_ptr<rule_t> &r = j.first->second;
1515  if (j.second)
1516  {
1517  r = ref_ptr<rule_t>(rule);
1518  r->targets = string_list(1, *i);
1519  continue;
1520  }
1521  if (!r->script.empty())
1522  {
1523  std::cerr << "Failed to load rules: " << *i
1524  << " cannot be the target of several rules" << std::endl;
1525  exit(EXIT_FAILURE);
1526  }
1527  assert(r->targets.size() == 1 && r->targets.front() == *i);
1528  merge_rule(*r, rule);
1529  }
1530 
1531  for (string_list::const_iterator i = targets.begin(),
1532  i_end = targets.end(); i != i_end; ++i)
1533  {
1535  if (dep->targets.empty()) dep->targets.push_back(*i);
1536  dep->deps.insert(rule.deps.begin(), rule.deps.end());
1537  }
1538 }
1539 
1540 /**
1541  * Register a specific rule with a nonempty script:
1542  *
1543  * - Check that none of the targets already has an associated rule.
1544  * - Create a single shared rule and associate it to all the targets.
1545  * - Merge the prerequisites of all the targets into a single set and
1546  * add the prerequisites of the rule to it. (The preexisting
1547  * prerequisites, if any, come from a previous run.)
1548  */
1549 static void register_scripted_rule(rule_t const &rule)
1550 {
1551  ref_ptr<rule_t> r(rule);
1552  for (string_list::const_iterator i = rule.targets.begin(),
1553  i_end = rule.targets.end(); i != i_end; ++i)
1554  {
1555  std::pair<rule_map::iterator, bool> j =
1556  specific_rules.insert(std::make_pair(*i, r));
1557  if (j.second) continue;
1558  std::cerr << "Failed to load rules: " << *i
1559  << " cannot be the target of several rules" << std::endl;
1560  exit(EXIT_FAILURE);
1561  }
1562 
1564  dep->targets = rule.targets;
1565  dep->deps.insert(rule.deps.begin(), rule.deps.end());
1566  for (string_list::const_iterator i = rule.targets.begin(),
1567  i_end = rule.targets.end(); i != i_end; ++i)
1568  {
1570  dep->deps.insert(d->deps.begin(), d->deps.end());
1571  d = dep;
1572  }
1573 }
1574 
1575 /**
1576  * Read a rule starting with target @a first, if nonempty.
1577  * Store into #generic_rules or #specific_rules depending on its genericity.
1578  */
1579 static void load_rule(std::istream &in, std::string const &first)
1580 {
1581  DEBUG_open << "Reading rule for target " << first << "... ";
1582  if (false)
1583  {
1584  error:
1585  DEBUG_close << "failed\n";
1586  std::cerr << "Failed to load rules: syntax error" << std::endl;
1587  exit(EXIT_FAILURE);
1588  }
1589  rule_t rule;
1590 
1591  // Read targets and check genericity.
1592  string_list targets;
1593  if (!read_words(in, targets)) goto error;
1594  if (!first.empty()) targets.push_front(first);
1595  else if (targets.empty()) goto error;
1596  else DEBUG << "actual target: " << targets.front() << std::endl;
1597  bool generic = false;
1598  normalize_list(targets, "", "");
1599  for (string_list::const_iterator i = targets.begin(),
1600  i_end = targets.end(); i != i_end; ++i)
1601  {
1602  if (i->empty()) goto error;
1603  if ((i->find('%') != std::string::npos) != generic)
1604  {
1605  if (i == targets.begin()) generic = true;
1606  else goto error;
1607  }
1608  }
1609  std::swap(rule.targets, targets);
1610  skip_spaces(in);
1611  if (in.get() != ':') goto error;
1612 
1613  bool assignment = false;
1614 
1615  // Read dependencies.
1616  {
1617  string_list v;
1618  if (expect_token(in, Word))
1619  {
1620  std::string d = read_word(in);
1621  if (int tok = expect_token(in, Equal | Plusequal))
1622  {
1623  if (!read_words(in, v)) goto error;
1624  assign_t &a = rule.assigns[d];
1625  a.append = tok == Plusequal;
1626  a.value.swap(v);
1627  assignment = true;
1628  goto end_line;
1629  }
1630  v.push_back(d);
1631  }
1632 
1633  if (!read_words(in, v)) goto error;
1634  normalize_list(v, "", "");
1635  rule.deps.swap(v);
1636 
1637  if (expect_token(in, Pipe))
1638  {
1639  if (!read_words(in, v)) goto error;
1640  normalize_list(v, "", "");
1641  rule.wdeps.swap(v);
1642  }
1643  }
1644 
1645  end_line:
1646  skip_spaces(in);
1647  if (!skip_eol(in, true)) goto error;
1648 
1649  // Read script.
1650  std::ostringstream buf;
1651  while (true)
1652  {
1653  char c = in.get();
1654  if (!in.good()) break;
1655  if (c == '\t' || c == ' ')
1656  {
1657  in.get(*buf.rdbuf());
1658  if (in.fail() && !in.eof()) in.clear();
1659  }
1660  else if (c == '\r' || c == '\n')
1661  buf << c;
1662  else
1663  {
1664  in.putback(c);
1665  break;
1666  }
1667  }
1668  rule.script = buf.str();
1669 
1670  // Register phony targets.
1671  if (rule.targets.front() == ".PHONY")
1672  {
1673  for (string_list::const_iterator i = rule.deps.begin(),
1674  i_end = rule.deps.end(); i != i_end; ++i)
1675  {
1676  status[*i].status = Todo;
1677  }
1678  return;
1679  }
1680 
1681  // Add generic rules to the correct set.
1682  if (generic)
1683  {
1684  if (assignment) goto error;
1685  generic_rules.push_back(rule);
1686  return;
1687  }
1688 
1689  if (!rule.script.empty())
1690  {
1691  if (assignment) goto error;
1692  register_scripted_rule(rule);
1693  }
1694  else
1695  {
1696  // Swap away the targets to avoid costly copies when registering.
1697  string_list targets;
1698  std::swap(rule.targets, targets);
1699  register_transparent_rule(rule, targets);
1700  std::swap(rule.targets, targets);
1701  }
1702 
1703  // If there is no default target yet, mark it as such.
1704  if (first_target.empty())
1705  first_target = rule.targets.front();
1706 }
1707 
1708 /**
1709  * Load rules from @a remakefile.
1710  * If some rules have dependencies and non-generic targets, add these
1711  * dependencies to the targets.
1712  */
1713 static void load_rules(std::string const &remakefile)
1714 {
1715  DEBUG_open << "Loading rules... ";
1716  if (false)
1717  {
1718  error:
1719  std::cerr << "Failed to load rules: syntax error" << std::endl;
1720  exit(EXIT_FAILURE);
1721  }
1722  std::ifstream in(remakefile.c_str());
1723  if (!in.good())
1724  {
1725  std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1726  exit(EXIT_FAILURE);
1727  }
1728  skip_empty(in);
1729 
1730  string_list options;
1731 
1732  // Read rules
1733  while (in.good())
1734  {
1735  char c = in.peek();
1736  if (c == '#')
1737  {
1738  while (in.get() != '\n') {}
1739  skip_empty(in);
1740  continue;
1741  }
1742  if (c == ' ' || c == '\t') goto error;
1743  if (expect_token(in, Word))
1744  {
1745  std::string name = read_word(in);
1746  if (name.empty()) goto error;
1747  if (int tok = expect_token(in, Equal | Plusequal))
1748  {
1749  DEBUG << "Assignment to variable " << name << std::endl;
1750  string_list value;
1751  if (!read_words(in, value)) goto error;
1752  string_list &dest =
1753  *(name == ".OPTIONS" ? &options : &variables[name]);
1754  if (tok == Equal) dest.swap(value);
1755  else dest.splice(dest.end(), value);
1756  if (!skip_eol(in, true)) goto error;
1757  }
1758  else load_rule(in, name);
1759  }
1760  else load_rule(in, std::string());
1761  }
1762 
1763  // Set actual options.
1764  for (string_list::const_iterator i = options.begin(),
1765  i_end = options.end(); i != i_end; ++i)
1766  {
1767  if (*i == "variable-propagation") propagate_vars = true;
1768  else
1769  {
1770  std::cerr << "Failed to load rules: unrecognized option" << std::endl;
1771  exit(EXIT_FAILURE);
1772  }
1773  }
1774 }
1775 
1776 /** @} */
1777 
1778 /**
1779  * @defgroup rules Rule resolution
1780  *
1781  * @{
1782  */
1783 
1784 static void merge_rule(rule_t &dest, rule_t const &src)
1785 {
1786  dest.deps.insert(dest.deps.end(), src.deps.begin(), src.deps.end());
1787  dest.wdeps.insert(dest.wdeps.end(), src.wdeps.begin(), src.wdeps.end());
1788  for (assign_map::const_iterator i = src.assigns.begin(),
1789  i_end = src.assigns.end(); i != i_end; ++i)
1790  {
1791  if (!i->second.append)
1792  {
1793  new_assign:
1794  dest.assigns[i->first] = i->second;
1795  continue;
1796  }
1797  assign_map::iterator j = dest.assigns.find(i->first);
1798  if (j == dest.assigns.end()) goto new_assign;
1799  j->second.value.insert(j->second.value.end(),
1800  i->second.value.begin(), i->second.value.end());
1801  }
1802 }
1803 
1804 /**
1805  * Substitute a pattern into a list of strings.
1806  */
1807 static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1808 {
1809  for (string_list::const_iterator i = src.begin(),
1810  i_end = src.end(); i != i_end; ++i)
1811  {
1812  size_t pos = i->find('%');
1813  if (pos == std::string::npos) dst.push_back(*i);
1814  else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1815  }
1816 }
1817 
1818 /**
1819  * Find a generic rule matching @a target:
1820  * - the one leading to shorter matches has priority,
1821  * - among equivalent rules, the earliest one has priority.
1822  */
1823 static void find_generic_rule(job_t &job, std::string const &target)
1824 {
1825  size_t tlen = target.length(), plen = tlen + 1;
1826  for (rule_list::const_iterator i = generic_rules.begin(),
1827  i_end = generic_rules.end(); i != i_end; ++i)
1828  {
1829  for (string_list::const_iterator j = i->targets.begin(),
1830  j_end = i->targets.end(); j != j_end; ++j)
1831  {
1832  size_t len = j->length();
1833  if (tlen < len) continue;
1834  if (plen <= tlen - (len - 1)) continue;
1835  size_t pos = j->find('%');
1836  if (pos == std::string::npos) continue;
1837  size_t len2 = len - (pos + 1);
1838  if (j->compare(0, pos, target, 0, pos) ||
1839  j->compare(pos + 1, len2, target, tlen - len2, len2))
1840  continue;
1841  plen = tlen - (len - 1);
1842  job.stem = target.substr(pos, plen);
1843  job.rule = rule_t();
1844  job.rule.script = i->script;
1845  substitute_pattern(job.stem, i->targets, job.rule.targets);
1846  substitute_pattern(job.stem, i->deps, job.rule.deps);
1847  substitute_pattern(job.stem, i->wdeps, job.rule.wdeps);
1848  break;
1849  }
1850  }
1851 }
1852 
1853 /**
1854  * Find a specific rule matching @a target. Return a generic one otherwise.
1855  * If there is both a specific rule with an empty script and a generic rule, the
1856  * generic one is returned after adding the dependencies of the specific one.
1857  */
1858 static void find_rule(job_t &job, std::string const &target)
1859 {
1860  rule_map::const_iterator i = specific_rules.find(target),
1861  i_end = specific_rules.end();
1862  // If there is a specific rule with a script, return it.
1863  if (i != i_end && !i->second->script.empty())
1864  {
1865  job.rule = *i->second;
1866  return;
1867  }
1868  find_generic_rule(job, target);
1869  // If there is no generic rule, return the specific rule (no script), if any.
1870  if (job.rule.targets.empty())
1871  {
1872  if (i != i_end)
1873  {
1874  job.rule = *i->second;
1875  return;
1876  }
1877  }
1878  // Optimize the lookup when there is only one target (already looked up).
1879  if (job.rule.targets.size() == 1)
1880  {
1881  if (i == i_end) return;
1882  merge_rule(job.rule, *i->second);
1883  return;
1884  }
1885  // Add the dependencies of the specific rules of every target to the
1886  // generic rule. If any of those rules has a nonempty script, error out.
1887  for (string_list::const_iterator j = job.rule.targets.begin(),
1888  j_end = job.rule.targets.end(); j != j_end; ++j)
1889  {
1890  i = specific_rules.find(*j);
1891  if (i == i_end) continue;
1892  if (!i->second->script.empty()) return;
1893  merge_rule(job.rule, *i->second);
1894  }
1895 }
1896 
1897 /** @} */
1898 
1899 /**
1900  * @defgroup status Target status
1901  *
1902  * @{
1903  */
1904 
1905 /**
1906  * Compute and memoize the status of @a target:
1907  * - if the file does not exist, the target is obsolete,
1908  * - if any dependency is obsolete or younger than the file, it is obsolete,
1909  * - otherwise it is up-to-date.
1910  *
1911  * @note For rules with multiple targets, all the targets share the same
1912  * status. (If one is obsolete, they all are.) The second rule above
1913  * is modified in that case: the latest target is chosen, not the oldest!
1914  */
1915 static status_t const &get_status(std::string const &target)
1916 {
1917  std::pair<status_map::iterator,bool> i =
1918  status.insert(std::make_pair(target, status_t()));
1919  status_t &ts = i.first->second;
1920  if (!i.second) return ts;
1921  DEBUG_open << "Checking status of " << target << "... ";
1922  dependency_map::const_iterator j = dependencies.find(target);
1923  if (j == dependencies.end())
1924  {
1925  struct stat s;
1926  if (stat(target.c_str(), &s) != 0)
1927  {
1928  DEBUG_close << "missing\n";
1929  ts.status = Todo;
1930  ts.last = 0;
1931  return ts;
1932  }
1933  DEBUG_close << "up-to-date\n";
1934  ts.status = Uptodate;
1935  ts.last = s.st_mtime;
1936  return ts;
1937  }
1938  dependency_t const &dep = *j->second;
1939  status_e st = Uptodate;
1940  time_t latest = 0;
1941  for (string_list::const_iterator k = dep.targets.begin(),
1942  k_end = dep.targets.end(); k != k_end; ++k)
1943  {
1944  struct stat s;
1945  if (stat(k->c_str(), &s) != 0)
1946  {
1947  if (st == Uptodate) DEBUG_close << *k << " missing\n";
1948  s.st_mtime = 0;
1949  st = Todo;
1950  }
1951  status[*k].last = s.st_mtime;
1952  if (s.st_mtime > latest) latest = s.st_mtime;
1953  }
1954  if (st != Uptodate) goto update;
1955  for (string_set::const_iterator k = dep.deps.begin(),
1956  k_end = dep.deps.end(); k != k_end; ++k)
1957  {
1958  status_t const &ts_ = get_status(*k);
1959  if (latest < ts_.last)
1960  {
1961  DEBUG_close << "older than " << *k << std::endl;
1962  st = Todo;
1963  goto update;
1964  }
1965  if (ts_.status != Uptodate && st != Recheck)
1966  {
1967  DEBUG << "obsolete dependency " << *k << std::endl;
1968  st = Recheck;
1969  }
1970  }
1971  if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
1972  update:
1973  for (string_list::const_iterator k = dep.targets.begin(),
1974  k_end = dep.targets.end(); k != k_end; ++k)
1975  {
1976  status[*k].status = st;
1977  }
1978  return ts;
1979 }
1980 
1981 /**
1982  * Change the status of @a target to #Remade or #Uptodate depending on whether
1983  * its modification time changed.
1984  */
1985 static void update_status(std::string const &target)
1986 {
1987  DEBUG_open << "Rechecking status of " << target << "... ";
1988  status_map::iterator i = status.find(target);
1989  assert(i != status.end());
1990  status_t &ts = i->second;
1991  ts.status = Remade;
1992  if (ts.last >= now)
1993  {
1994  DEBUG_close << "possibly remade\n";
1995  return;
1996  }
1997  struct stat s;
1998  if (stat(target.c_str(), &s) != 0)
1999  {
2000  DEBUG_close << "missing\n";
2001  ts.last = 0;
2002  }
2003  else if (s.st_mtime != ts.last)
2004  {
2005  DEBUG_close << "remade\n";
2006  ts.last = s.st_mtime;
2007  }
2008  else
2009  {
2010  DEBUG_close << "unchanged\n";
2011  ts.status = Uptodate;
2012  }
2013 }
2014 
2015 /**
2016  * Check whether all the prerequisites of @a target ended being up-to-date.
2017  */
2018 static bool still_need_rebuild(std::string const &target)
2019 {
2020  DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
2021  status_map::const_iterator i = status.find(target);
2022  assert(i != status.end());
2023  if (i->second.status != Recheck) return true;
2024  dependency_map::const_iterator j = dependencies.find(target);
2025  assert(j != dependencies.end());
2026  dependency_t const &dep = *j->second;
2027  for (string_set::const_iterator k = dep.deps.begin(),
2028  k_end = dep.deps.end(); k != k_end; ++k)
2029  {
2030  if (status[*k].status != Uptodate) return true;
2031  }
2032  for (string_list::const_iterator k = dep.targets.begin(),
2033  k_end = dep.targets.end(); k != k_end; ++k)
2034  {
2035  status[*k].status = Uptodate;
2036  }
2037  DEBUG_close << "no longer obsolete\n";
2038  return false;
2039 }
2040 
2041 /** @} */
2042 
2043 /**
2044  * @defgroup server Server
2045  *
2046  * @{
2047  */
2048 
2049 /**
2050  * Handle job completion.
2051  */
2052 static void complete_job(int job_id, bool success)
2053 {
2054  DEBUG << "Completing job " << job_id << '\n';
2055  job_map::iterator i = jobs.find(job_id);
2056  assert(i != jobs.end());
2057  string_list const &targets = i->second.rule.targets;
2058  if (success)
2059  {
2060  if (show_targets) std::cout << "Finished";
2061  for (string_list::const_iterator j = targets.begin(),
2062  j_end = targets.end(); j != j_end; ++j)
2063  {
2064  update_status(*j);
2065  if (show_targets) std::cout << ' ' << *j;
2066  }
2067  if (show_targets) std::cout << std::endl;
2068  }
2069  else
2070  {
2071  std::cerr << "Failed to build";
2072  for (string_list::const_iterator j = targets.begin(),
2073  j_end = targets.end(); j != j_end; ++j)
2074  {
2075  status[*j].status = Failed;
2076  std::cerr << ' ' << *j;
2077  remove(j->c_str());
2078  }
2079  std::cerr << std::endl;
2080  }
2081  jobs.erase(i);
2082 }
2083 
2084 /**
2085  * Return the script obtained by substituting variables.
2086  */
2087 static std::string prepare_script(job_t const &job)
2088 {
2089  std::string const &s = job.rule.script;
2090  std::istringstream in(s);
2091  std::ostringstream out;
2092  size_t len = s.size();
2093 
2094  while (!in.eof())
2095  {
2096  size_t pos = in.tellg(), p = s.find('$', pos);
2097  if (p == std::string::npos || p == len - 1) p = len;
2098  out.write(&s[pos], p - pos);
2099  if (p == len) break;
2100  ++p;
2101  switch (s[p])
2102  {
2103  case '$':
2104  out << '$';
2105  in.seekg(p + 1);
2106  break;
2107  case '<':
2108  if (!job.rule.deps.empty())
2109  out << job.rule.deps.front();
2110  in.seekg(p + 1);
2111  break;
2112  case '^':
2113  {
2114  bool first = true;
2115  for (string_list::const_iterator i = job.rule.deps.begin(),
2116  i_end = job.rule.deps.end(); i != i_end; ++i)
2117  {
2118  if (first) first = false;
2119  else out << ' ';
2120  out << *i;
2121  }
2122  in.seekg(p + 1);
2123  break;
2124  }
2125  case '@':
2126  assert(!job.rule.targets.empty());
2127  out << job.rule.targets.front();
2128  in.seekg(p + 1);
2129  break;
2130  case '*':
2131  out << job.stem;
2132  in.seekg(p + 1);
2133  break;
2134  case '(':
2135  {
2136  in.seekg(p - 1);
2137  bool first = true;
2138  input_generator gen(in, &job.vars, true);
2139  while (true)
2140  {
2141  std::string w;
2142  input_status s = gen.next(w);
2143  if (s == SyntaxError)
2144  {
2145  // TODO
2146  return "false";
2147  }
2148  if (s == Eof) break;
2149  if (first) first = false;
2150  else out << ' ';
2151  out << w;
2152  }
2153  break;
2154  }
2155  default:
2156  // Let dollars followed by an unrecognized character
2157  // go through. This differs from Make, which would
2158  // use a one-letter variable.
2159  out << '$';
2160  in.seekg(p);
2161  }
2162  }
2163 
2164  return out.str();
2165 }
2166 
2167 /**
2168  * Execute the script from @a rule.
2169  */
2170 static status_e run_script(int job_id, job_t const &job)
2171 {
2173  dep->targets = job.rule.targets;
2174  dep->deps.insert(job.rule.deps.begin(), job.rule.deps.end());
2175  if (show_targets) std::cout << "Building";
2176  for (string_list::const_iterator i = job.rule.targets.begin(),
2177  i_end = job.rule.targets.end(); i != i_end; ++i)
2178  {
2179  dependencies[*i] = dep;
2180  if (show_targets) std::cout << ' ' << *i;
2181  }
2182  if (show_targets) std::cout << std::endl;
2183 
2184  std::string script = prepare_script(job);
2185 
2186  std::ostringstream job_id_buf;
2187  job_id_buf << job_id;
2188  std::string job_id_ = job_id_buf.str();
2189 
2190  DEBUG_open << "Starting script for job " << job_id << "... ";
2191  if (script.empty())
2192  {
2193  DEBUG_close << "no script\n";
2194  complete_job(job_id, true);
2195  return Remade;
2196  }
2197 
2198  if (false)
2199  {
2200  error:
2201  DEBUG_close << "failed\n";
2202  complete_job(job_id, false);
2203  return Failed;
2204  }
2205 
2206 #ifdef WINDOWS
2207  HANDLE pfd[2];
2208  if (false)
2209  {
2210  error2:
2211  CloseHandle(pfd[0]);
2212  CloseHandle(pfd[1]);
2213  goto error;
2214  }
2215  if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
2216  goto error;
2217  if (!SetHandleInformation(pfd[0], HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT))
2218  goto error2;
2219  STARTUPINFO si;
2220  ZeroMemory(&si, sizeof(STARTUPINFO));
2221  si.cb = sizeof(STARTUPINFO);
2222  si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
2223  si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
2224  si.hStdInput = pfd[0];
2225  si.dwFlags |= STARTF_USESTDHANDLES;
2226  PROCESS_INFORMATION pi;
2227  ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
2228  if (!SetEnvironmentVariable("REMAKE_JOB_ID", job_id_.c_str()))
2229  goto error2;
2230  char const *argv = echo_scripts ? "SH.EXE -e -s -v" : "SH.EXE -e -s";
2231  if (!CreateProcess(NULL, (char *)argv, NULL, NULL,
2232  true, 0, NULL, NULL, &si, &pi))
2233  {
2234  goto error2;
2235  }
2236  CloseHandle(pi.hThread);
2237  DWORD len = script.length(), wlen;
2238  if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
2239  std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2240  CloseHandle(pfd[0]);
2241  CloseHandle(pfd[1]);
2242  ++running_jobs;
2243  job_pids[pi.hProcess] = job_id;
2244  return Running;
2245 #else
2246  int pfd[2];
2247  if (false)
2248  {
2249  error2:
2250  close(pfd[0]);
2251  close(pfd[1]);
2252  goto error;
2253  }
2254  if (pipe(pfd) == -1)
2255  goto error;
2256  if (setenv("REMAKE_JOB_ID", job_id_.c_str(), 1))
2257  goto error2;
2258  if (pid_t pid = vfork())
2259  {
2260  if (pid == -1) goto error2;
2261  ssize_t len = script.length();
2262  if (write(pfd[1], script.c_str(), len) < len)
2263  std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2264  close(pfd[0]);
2265  close(pfd[1]);
2266  ++running_jobs;
2267  job_pids[pid] = job_id;
2268  return Running;
2269  }
2270  // Child process starts here. Notice the use of vfork above.
2271  char const *argv[5] = { "sh", "-e", "-s", NULL, NULL };
2272  if (echo_scripts) argv[3] = "-v";
2273  close(pfd[1]);
2274  if (pfd[0] != 0)
2275  {
2276  dup2(pfd[0], 0);
2277  close(pfd[0]);
2278  }
2279  execve("/bin/sh", (char **)argv, environ);
2280  _exit(EXIT_FAILURE);
2281 #endif
2282 }
2283 
2284 /**
2285  * Create a job for @a target according to the loaded rules.
2286  * Mark all the targets from the rule as running and reset their dependencies.
2287  * Inherit variables from @a current, if enabled.
2288  * If the rule has dependencies, create a new client to build them just
2289  * before @a current, and change @a current so that it points to it.
2290  */
2291 static status_e start(std::string const &target, client_list::iterator &current)
2292 {
2293  int job_id = job_counter++;
2294  DEBUG_open << "Starting job " << job_id << " for " << target << "... ";
2295  job_t &job = jobs[job_id];
2296  find_rule(job, target);
2297  if (job.rule.targets.empty())
2298  {
2299  status[target].status = Failed;
2300  DEBUG_close << "failed\n";
2301  std::cerr << "No rule for building " << target << std::endl;
2302  return Failed;
2303  }
2304  for (string_list::const_iterator i = job.rule.targets.begin(),
2305  i_end = job.rule.targets.end(); i != i_end; ++i)
2306  {
2307  status[*i].status = Running;
2308  }
2309  if (propagate_vars) job.vars = current->vars;
2310  for (assign_map::const_iterator i = job.rule.assigns.begin(),
2311  i_end = job.rule.assigns.end(); i != i_end; ++i)
2312  {
2313  std::pair<variable_map::iterator, bool> k =
2314  job.vars.insert(std::make_pair(i->first, string_list()));
2315  string_list &v = k.first->second;
2316  if (i->second.append)
2317  {
2318  if (k.second)
2319  {
2320  variable_map::const_iterator j = variables.find(i->first);
2321  if (j != variables.end()) v = j->second;
2322  }
2323  }
2324  else if (!k.second) v.clear();
2325  v.insert(v.end(), i->second.value.begin(), i->second.value.end());
2326  }
2327  if (!job.rule.deps.empty() || !job.rule.wdeps.empty())
2328  {
2329  current = clients.insert(current, client_t());
2330  current->job_id = job_id;
2331  current->pending = job.rule.deps;
2332  current->pending.insert(current->pending.end(),
2333  job.rule.wdeps.begin(), job.rule.wdeps.end());
2334  if (propagate_vars) current->vars = job.vars;
2335  current->delayed = true;
2336  return Recheck;
2337  }
2338  return run_script(job_id, job);
2339 }
2340 
2341 /**
2342  * Send a reply to a client then remove it.
2343  * If the client was a dependency client, start the actual script.
2344  */
2345 static void complete_request(client_t &client, bool success)
2346 {
2347  DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
2348  if (client.delayed)
2349  {
2350  assert(client.socket == INVALID_SOCKET);
2351  if (success)
2352  {
2353  job_map::const_iterator i = jobs.find(client.job_id);
2354  assert(i != jobs.end());
2355  if (still_need_rebuild(i->second.rule.targets.front()))
2356  run_script(client.job_id, i->second);
2357  else complete_job(client.job_id, true);
2358  }
2359  else complete_job(client.job_id, false);
2360  }
2361  else if (client.socket != INVALID_SOCKET)
2362  {
2363  char res = success ? 1 : 0;
2364  send(client.socket, &res, 1, MSG_NOSIGNAL);
2365  #ifdef WINDOWS
2366  closesocket(client.socket);
2367  #else
2368  close(client.socket);
2369  #endif
2370  --waiting_jobs;
2371  }
2372 
2373  if (client.job_id < 0 && !success) build_failure = true;
2374 }
2375 
2376 /**
2377  * Return whether there are slots for starting new jobs.
2378  */
2379 static bool has_free_slots()
2380 {
2381  if (max_active_jobs <= 0) return true;
2383 }
2384 
2385 /**
2386  * Handle client requests:
2387  * - check for running targets that have finished,
2388  * - start as many pending targets as allowed,
2389  * - complete the request if there are neither running nor pending targets
2390  * left or if any of them failed.
2391  *
2392  * @return true if some child processes are still running.
2393  *
2394  * @post If there are pending requests, at least one child process is running.
2395  *
2396  * @invariant New free slots cannot appear during a run, since the only way to
2397  * decrease #running_jobs is #finalize_job and the only way to
2398  * increase #waiting_jobs is #accept_client. None of these functions
2399  * are called during a run. So breaking out as soon as there no free
2400  * slots left is fine.
2401  */
2402 static bool handle_clients()
2403 {
2404  DEBUG_open << "Handling client requests... ";
2405  restart:
2406  bool need_restart = false;
2407 
2408  for (client_list::iterator i = clients.begin(), i_next = i,
2409  i_end = clients.end(); i != i_end && has_free_slots(); i = i_next)
2410  {
2411  ++i_next;
2412  DEBUG_open << "Handling client from job " << i->job_id << "... ";
2413 
2414  // Remove running targets that have finished.
2415  for (string_set::iterator j = i->running.begin(), j_next = j,
2416  j_end = i->running.end(); j != j_end; j = j_next)
2417  {
2418  ++j_next;
2419  status_map::const_iterator k = status.find(*j);
2420  assert(k != status.end());
2421  switch (k->second.status)
2422  {
2423  case Running:
2424  break;
2425  case Failed:
2426  i->failed = true;
2427  if (!keep_going) goto complete;
2428  // no break
2429  case Uptodate:
2430  case Remade:
2431  i->running.erase(j);
2432  break;
2433  case Recheck:
2434  case Todo:
2435  assert(false);
2436  }
2437  }
2438 
2439  // Start pending targets.
2440  while (!i->pending.empty())
2441  {
2442  std::string target = i->pending.front();
2443  i->pending.pop_front();
2444  switch (get_status(target).status)
2445  {
2446  case Running:
2447  i->running.insert(target);
2448  break;
2449  case Failed:
2450  pending_failed:
2451  i->failed = true;
2452  if (!keep_going) goto complete;
2453  // no break
2454  case Uptodate:
2455  case Remade:
2456  break;
2457  case Recheck:
2458  case Todo:
2459  client_list::iterator j = i;
2460  switch (start(target, i))
2461  {
2462  case Failed:
2463  goto pending_failed;
2464  case Running:
2465  // A shell was started, check for free slots.
2466  j->running.insert(target);
2467  if (!has_free_slots()) return true;
2468  break;
2469  case Recheck:
2470  // Switch to the dependency client that was inserted.
2471  j->running.insert(target);
2472  i_next = j;
2473  break;
2474  case Remade:
2475  // Nothing to run.
2476  need_restart = true;
2477  break;
2478  default:
2479  assert(false);
2480  }
2481  }
2482  }
2483 
2484  // Try to complete the request.
2485  // (This might start a new job if it was a dependency client.)
2486  if (i->running.empty() || i->failed)
2487  {
2488  complete:
2489  complete_request(*i, !i->failed);
2490  DEBUG_close << (i->failed ? "failed\n" : "finished\n");
2491  clients.erase(i);
2492  need_restart = true;
2493  }
2494  }
2495 
2496  if (running_jobs != waiting_jobs) return true;
2497  if (running_jobs == 0 && clients.empty()) return false;
2498  if (need_restart) goto restart;
2499 
2500  // There is a circular dependency.
2501  // Try to break it by completing one of the requests.
2502  assert(!clients.empty());
2503  std::cerr << "Circular dependency detected" << std::endl;
2504  client_list::iterator i = clients.begin();
2505  complete_request(*i, false);
2506  clients.erase(i);
2507  goto restart;
2508 }
2509 
2510 /**
2511  * Create a named unix socket that listens for build requests. Also set
2512  * the REMAKE_SOCKET environment variable that will be inherited by all
2513  * the job scripts.
2514  */
2515 static void create_server()
2516 {
2517  if (false)
2518  {
2519  error:
2520  perror("Failed to create server");
2521 #ifndef WINDOWS
2522  error2:
2523 #endif
2524  exit(EXIT_FAILURE);
2525  }
2526  DEBUG_open << "Creating server... ";
2527 
2528 #ifdef WINDOWS
2529  // Prepare a windows socket.
2530  struct sockaddr_in socket_addr;
2531  socket_addr.sin_family = AF_INET;
2532  socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2533  socket_addr.sin_port = 0;
2534 
2535  // Create and listen to the socket.
2536  socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2537  if (socket_fd == INVALID_SOCKET) goto error;
2538  if (!SetHandleInformation((HANDLE)socket_fd, HANDLE_FLAG_INHERIT, 0))
2539  goto error;
2540  if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2541  goto error;
2542  int len = sizeof(sockaddr_in);
2543  if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
2544  goto error;
2545  std::ostringstream buf;
2546  buf << socket_addr.sin_port;
2547  if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
2548  goto error;
2549  if (listen(socket_fd, 1000)) goto error;
2550 #else
2551  // Set signal handlers for SIGCHLD and SIGINT.
2552  // Block SIGCHLD (unblocked during select).
2553  sigset_t sigmask;
2554  sigemptyset(&sigmask);
2555  sigaddset(&sigmask, SIGCHLD);
2556  if (sigprocmask(SIG_BLOCK, &sigmask, NULL) == -1) goto error;
2557  struct sigaction sa;
2558  sa.sa_flags = 0;
2559  sigemptyset(&sa.sa_mask);
2560  sa.sa_handler = &sigchld_handler;
2561  if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
2562  sa.sa_handler = &sigint_handler;
2563  if (sigaction(SIGINT, &sa, NULL) == -1) goto error;
2564 
2565  // Prepare a named unix socket in temporary directory.
2566  socket_name = tempnam(NULL, "rmk-");
2567  if (!socket_name) goto error2;
2568  struct sockaddr_un socket_addr;
2569  size_t len = strlen(socket_name);
2570  if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
2571  socket_addr.sun_family = AF_UNIX;
2572  strcpy(socket_addr.sun_path, socket_name);
2573  len += sizeof(socket_addr.sun_family);
2574  if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
2575 
2576  // Create and listen to the socket.
2577 #ifdef LINUX
2578  socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
2579  if (socket_fd == INVALID_SOCKET) goto error;
2580 #else
2581  socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2582  if (socket_fd == INVALID_SOCKET) goto error;
2583  if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
2584 #endif
2585  if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
2586  goto error;
2587  if (listen(socket_fd, 1000)) goto error;
2588 #endif
2589 }
2590 
2591 /**
2592  * Accept a connection from a client, get the job it spawned from,
2593  * get the targets, and mark them as dependencies of the job targets.
2594  */
2595 static void accept_client()
2596 {
2597  DEBUG_open << "Handling client request... ";
2598 
2599  // Accept connection.
2600 #ifdef WINDOWS
2601  socket_t fd = accept(socket_fd, NULL, NULL);
2602  if (fd == INVALID_SOCKET) return;
2603  if (!SetHandleInformation((HANDLE)fd, HANDLE_FLAG_INHERIT, 0))
2604  {
2605  error2:
2606  std::cerr << "Unexpected failure while setting connection with client" << std::endl;
2607  closesocket(fd);
2608  return;
2609  }
2610  // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
2611  u_long nbio = 0;
2612  if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
2613 #elif defined(LINUX)
2614  int fd = accept4(socket_fd, NULL, NULL, SOCK_CLOEXEC);
2615  if (fd < 0) return;
2616 #else
2617  int fd = accept(socket_fd, NULL, NULL);
2618  if (fd < 0) return;
2619  if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
2620 #endif
2621  clients.push_front(client_t());
2622  client_list::iterator proc = clients.begin();
2623 
2624  if (false)
2625  {
2626  error:
2627  DEBUG_close << "failed\n";
2628  std::cerr << "Received an ill-formed client message" << std::endl;
2629  #ifdef WINDOWS
2630  closesocket(fd);
2631  #else
2632  close(fd);
2633  #endif
2634  clients.erase(proc);
2635  return;
2636  }
2637 
2638  // Receive message. Stop when encountering two nuls in a row.
2639  std::vector<char> buf;
2640  size_t len = 0;
2641  while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
2642  {
2643  buf.resize(len + 1024);
2644  ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
2645  if (l <= 0) goto error;
2646  len += l;
2647  }
2648 
2649  // Parse job that spawned the client.
2650  int job_id;
2651  memcpy(&job_id, &buf[0], sizeof(int));
2652  proc->socket = fd;
2653  proc->job_id = job_id;
2654  job_map::const_iterator i = jobs.find(job_id);
2655  if (i == jobs.end()) goto error;
2656  DEBUG << "receiving request from job " << job_id << std::endl;
2657  if (propagate_vars) proc->vars = i->second.vars;
2658 
2659  // Parse the targets and the variable assignments.
2660  // Mark the targets as dependencies of the job targets.
2661  dependency_t &dep = *dependencies[i->second.rule.targets.front()];
2662  string_list *last_var = NULL;
2663  char const *p = &buf[0] + sizeof(int);
2664  while (true)
2665  {
2666  len = strlen(p);
2667  if (len == 0)
2668  {
2669  ++waiting_jobs;
2670  break;
2671  }
2672  switch (*p)
2673  {
2674  case 'T':
2675  {
2676  if (len == 1) goto error;
2677  std::string target(p + 1, p + len);
2678  DEBUG << "adding dependency " << target << " to job\n";
2679  proc->pending.push_back(target);
2680  dep.deps.insert(target);
2681  break;
2682  }
2683  case 'V':
2684  {
2685  if (len == 1) goto error;
2686  std::string var(p + 1, p + len);
2687  DEBUG << "adding variable " << var << " to job\n";
2688  last_var = &proc->vars[var];
2689  last_var->clear();
2690  break;
2691  }
2692  case 'W':
2693  {
2694  if (!last_var) goto error;
2695  last_var->push_back(std::string(p + 1, p + len));
2696  break;
2697  }
2698  default:
2699  goto error;
2700  }
2701  p += len + 1;
2702  }
2703 
2704  if (!propagate_vars && !proc->vars.empty())
2705  {
2706  std::cerr << "Assignments are ignored unless 'variable-propagation' is enabled" << std::endl;
2707  proc->vars.clear();
2708  }
2709 }
2710 
2711 /**
2712  * Handle child process exit status.
2713  */
2714 static void finalize_job(pid_t pid, bool res)
2715 {
2716  pid_job_map::iterator i = job_pids.find(pid);
2717  assert(i != job_pids.end());
2718  int job_id = i->second;
2719  job_pids.erase(i);
2720  --running_jobs;
2721  complete_job(job_id, res);
2722 }
2723 
2724 /**
2725  * Loop until all the jobs have finished.
2726  *
2727  * @post There are no client requests left, not even virtual ones.
2728  */
2729 static void server_loop()
2730 {
2731  while (handle_clients())
2732  {
2733  DEBUG_open << "Handling events... ";
2734  #ifdef WINDOWS
2735  size_t len = job_pids.size() + 1;
2736  HANDLE h[len];
2737  int num = 0;
2738  for (pid_job_map::const_iterator i = job_pids.begin(),
2739  i_end = job_pids.end(); i != i_end; ++i, ++num)
2740  {
2741  h[num] = i->first;
2742  }
2743  WSAEVENT aev = WSACreateEvent();
2744  h[num] = aev;
2745  WSAEventSelect(socket_fd, aev, FD_ACCEPT);
2746  DWORD w = WaitForMultipleObjects(len, h, false, INFINITE);
2747  WSAEventSelect(socket_fd, aev, 0);
2748  WSACloseEvent(aev);
2749  if (len <= w)
2750  continue;
2751  if (w == len - 1)
2752  {
2753  accept_client();
2754  continue;
2755  }
2756  pid_t pid = h[w];
2757  DWORD s = 0;
2758  bool res = GetExitCodeProcess(pid, &s) && s == 0;
2759  CloseHandle(pid);
2760  finalize_job(pid, res);
2761  #else
2762  sigset_t emptymask;
2763  sigemptyset(&emptymask);
2764  fd_set fdset;
2765  FD_ZERO(&fdset);
2766  FD_SET(socket_fd, &fdset);
2767  int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2768  if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2769  if (!got_SIGCHLD) continue;
2770  got_SIGCHLD = 0;
2771  pid_t pid;
2772  int status;
2773  while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2774  {
2775  bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2776  finalize_job(pid, res);
2777  }
2778  #endif
2779  }
2780 
2781  assert(clients.empty());
2782 }
2783 
2784 /**
2785  * Load dependencies and rules, listen to client requests, and loop until
2786  * all the requests have completed.
2787  * If Remakefile is obsolete, perform a first run with it only, then reload
2788  * the rules, and perform a second with the original clients.
2789  */
2790 static void server_mode(std::string const &remakefile, string_list const &targets)
2791 {
2793  load_rules(remakefile);
2794  create_server();
2795  if (get_status(remakefile).status != Uptodate)
2796  {
2797  clients.push_back(client_t());
2798  clients.back().pending.push_back(remakefile);
2799  server_loop();
2800  if (build_failure) goto early_exit;
2801  variables.clear();
2802  specific_rules.clear();
2803  generic_rules.clear();
2804  first_target.clear();
2805  load_rules(remakefile);
2806  }
2807  clients.push_back(client_t());
2808  if (!targets.empty()) clients.back().pending = targets;
2809  else if (!first_target.empty())
2810  clients.back().pending.push_back(first_target);
2811  server_loop();
2812  early_exit:
2813  close(socket_fd);
2814 #ifndef WINDOWS
2815  remove(socket_name);
2816  free(socket_name);
2817 #endif
2820  {
2821  std::cout << "remake: Leaving directory `" << prefix_dir << '\'' << std::endl;
2822  }
2823  exit(build_failure ? EXIT_FAILURE : EXIT_SUCCESS);
2824 }
2825 
2826 /** @} */
2827 
2828 /**
2829  * @defgroup client Client
2830  *
2831  * @{
2832  */
2833 
2834 /**
2835  * Connect to the server @a socket_name, send a request for building @a targets
2836  * with some @a variables, and exit with the status returned by the server.
2837  */
2838 static void client_mode(char *socket_name, string_list const &targets)
2839 {
2840  if (false)
2841  {
2842  error:
2843  perror("Failed to send targets to server");
2844  exit(EXIT_FAILURE);
2845  }
2846  if (targets.empty()) exit(EXIT_SUCCESS);
2847  DEBUG_open << "Connecting to server... ";
2848 
2849  // Connect to server.
2850 #ifdef WINDOWS
2851  struct sockaddr_in socket_addr;
2852  socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2853  if (socket_fd == INVALID_SOCKET) goto error;
2854  socket_addr.sin_family = AF_INET;
2855  socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2856  socket_addr.sin_port = atoi(socket_name);
2857  if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2858  goto error;
2859 #else
2860  struct sockaddr_un socket_addr;
2861  size_t len = strlen(socket_name);
2862  if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2863  socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2864  if (socket_fd == INVALID_SOCKET) goto error;
2865  socket_addr.sun_family = AF_UNIX;
2866  strcpy(socket_addr.sun_path, socket_name);
2867  if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2868  goto error;
2869 #ifdef MACOSX
2870  int set_option = 1;
2871  if (setsockopt(socket_fd, SOL_SOCKET, SO_NOSIGPIPE, &set_option, sizeof(set_option)))
2872  goto error;
2873 #endif
2874 #endif
2875 
2876  // Send current job id.
2877  char *id = getenv("REMAKE_JOB_ID");
2878  int job_id = id ? atoi(id) : -1;
2879  if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2880  goto error;
2881 
2882  // Send targets.
2883  for (string_list::const_iterator i = targets.begin(),
2884  i_end = targets.end(); i != i_end; ++i)
2885  {
2886  DEBUG_open << "Sending target " << *i << "... ";
2887  std::string s = 'T' + *i;
2888  ssize_t len = s.length() + 1;
2889  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2890  goto error;
2891  }
2892 
2893  // Send variables.
2894  for (variable_map::const_iterator i = variables.begin(),
2895  i_end = variables.end(); i != i_end; ++i)
2896  {
2897  DEBUG_open << "Sending variable " << i->first << "... ";
2898  std::string s = 'V' + i->first;
2899  ssize_t len = s.length() + 1;
2900  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2901  goto error;
2902  for (string_list::const_iterator j = i->second.begin(),
2903  j_end = i->second.end(); j != j_end; ++j)
2904  {
2905  std::string s = 'W' + *j;
2906  len = s.length() + 1;
2907  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2908  goto error;
2909  }
2910  }
2911 
2912  // Send terminating nul and wait for reply.
2913  char result = 0;
2914  if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2915  if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
2916  exit(result ? EXIT_SUCCESS : EXIT_FAILURE);
2917 }
2918 
2919 /** @} */
2920 
2921 /**
2922  * @defgroup ui User interface
2923  *
2924  * @{
2925  */
2926 
2927 /**
2928  * Display usage and exit with @a exit_status.
2929  */
2930 static void usage(int exit_status)
2931 {
2932  std::cerr << "Usage: remake [options] [target] ...\n"
2933  "Options\n"
2934  " -d Echo script commands.\n"
2935  " -d -d Print lots of debugging information.\n"
2936  " -f FILE Read FILE as Remakefile.\n"
2937  " -h, --help Print this message and exit.\n"
2938  " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
2939  " -k Keep going when some targets cannot be made.\n"
2940  " -r Look up targets from the dependencies on stdin.\n"
2941  " -s, --silent, --quiet Do not echo targets.\n";
2942  exit(exit_status);
2943 }
2944 
2945 /**
2946  * This program behaves in two different ways.
2947  *
2948  * - If the environment contains the REMAKE_SOCKET variable, the client
2949  * connects to this socket and sends to the server its build targets.
2950  * It exits once it receives the server reply.
2951  *
2952  * - Otherwise, it creates a server that waits for build requests. It
2953  * also creates a pseudo-client that requests the targets passed on the
2954  * command line.
2955  */
2956 int main(int argc, char *argv[])
2957 {
2958  std::string remakefile;
2959  string_list targets;
2960  bool literal_targets = false;
2961  bool indirect_targets = false;
2962 
2963  // Parse command-line arguments.
2964  for (int i = 1; i < argc; ++i)
2965  {
2966  std::string arg = argv[i];
2967  if (arg.empty()) usage(EXIT_FAILURE);
2968  if (literal_targets) goto new_target;
2969  if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
2970  if (arg == "-d")
2971  if (echo_scripts) debug.active = true;
2972  else echo_scripts = true;
2973  else if (arg == "-k" || arg =="--keep-going")
2974  keep_going = true;
2975  else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
2976  show_targets = false;
2977  else if (arg == "-r")
2978  indirect_targets = true;
2979  else if (arg == "-f")
2980  {
2981  if (++i == argc) usage(EXIT_FAILURE);
2982  remakefile = argv[i];
2983  }
2984  else if (arg == "--")
2985  literal_targets = true;
2986  else if (arg.compare(0, 2, "-j") == 0)
2987  max_active_jobs = atoi(arg.c_str() + 2);
2988  else if (arg.compare(0, 7, "--jobs=") == 0)
2989  max_active_jobs = atoi(arg.c_str() + 7);
2990  else
2991  {
2992  if (arg[0] == '-') usage(EXIT_FAILURE);
2993  if (arg.find('=') != std::string::npos)
2994  {
2995  std::istringstream in(arg);
2996  std::string name = read_word(in);
2997  if (name.empty() || !expect_token(in, Equal)) usage(EXIT_FAILURE);
2998  read_words(in, variables[name]);
2999  continue;
3000  }
3001  new_target:
3002  targets.push_back(arg);
3003  DEBUG << "New target: " << arg << '\n';
3004  }
3005  }
3006 
3007  init_working_dir();
3009 
3010  if (indirect_targets)
3011  {
3012  load_dependencies(std::cin);
3013  string_list l;
3014  targets.swap(l);
3015  if (l.empty() && !dependencies.empty())
3016  {
3017  l.push_back(dependencies.begin()->second->targets.front());
3018  }
3019  for (string_list::const_iterator i = l.begin(),
3020  i_end = l.end(); i != i_end; ++i)
3021  {
3022  dependency_map::const_iterator j = dependencies.find(*i);
3023  if (j == dependencies.end()) continue;
3024  dependency_t const &dep = *j->second;
3025  for (string_set::const_iterator k = dep.deps.begin(),
3026  k_end = dep.deps.end(); k != k_end; ++k)
3027  {
3028  targets.push_back(normalize(*k, working_dir, working_dir));
3029  }
3030  }
3031  dependencies.clear();
3032  }
3033 
3034 #ifdef WINDOWS
3035  WSADATA wsaData;
3036  if (WSAStartup(MAKEWORD(2,2), &wsaData))
3037  {
3038  std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
3039  return 1;
3040  }
3041 #endif
3042 
3043  // Run as client if REMAKE_SOCKET is present in the environment.
3044  if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
3045 
3046  // Otherwise run as server.
3047  if (remakefile.empty())
3048  {
3049  remakefile = "Remakefile";
3050  init_prefix_dir();
3051  }
3053  server_mode(remakefile, targets);
3054 }
3055 
3056 /** @} */
std::map< int, job_t > job_map
Definition: remake.cpp:567
static void create_server()
Definition: remake.cpp:2515
std::string const & input
Definition: remake.cpp:804
static bool still_need_rebuild(std::string const &target)
Definition: remake.cpp:2018
int depth
Definition: remake.cpp:755
T * operator->() const
Definition: remake.cpp:492
input_status next(std::string &)
Definition: remake.cpp:1362
int job_id
Job for which the built script called remake and spawned the client (negative for original clients)...
Definition: remake.cpp:587
static status_e run_script(int job_id, job_t const &job)
Definition: remake.cpp:2170
ref_ptr & operator=(ref_ptr const &p)
Definition: remake.cpp:479
status_e
Definition: remake.cpp:508
static void complete_request(client_t &client, bool success)
Definition: remake.cpp:2345
static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
Definition: remake.cpp:931
static generator * get_function(input_generator const &, std::string const &)
Definition: remake.cpp:1392
std::map< std::string, status_t > status_map
Definition: remake.cpp:527
static void find_rule(job_t &job, std::string const &target)
Definition: remake.cpp:1858
static void save_dependencies()
Definition: remake.cpp:1465
static void server_mode(std::string const &remakefile, string_list const &targets)
Definition: remake.cpp:2790
std::string pre
Definition: remake.cpp:1347
static void init_prefix_dir()
Definition: remake.cpp:875
#define DEBUG_open
Definition: remake.cpp:794
static char * socket_name
Definition: remake.cpp:693
static job_map jobs
Definition: remake.cpp:627
static bool propagate_vars
Definition: remake.cpp:734
std::ostream & operator()(bool o)
Definition: remake.cpp:767
static variable_map variables
Definition: remake.cpp:602
std::list< std::string > string_list
Definition: remake.cpp:455
string_list targets
Definition: remake.cpp:497
variable_map vars
Variables set on request.
Definition: remake.cpp:591
addprefix_generator(input_generator const &, bool &)
Definition: remake.cpp:1296
static int max_active_jobs
Definition: remake.cpp:644
escape_string(std::string const &s)
Definition: remake.cpp:805
bool delayed
Whether it is a dependency client and a script has to be started on request completion.
Definition: remake.cpp:592
Target is being rebuilt.
Definition: remake.cpp:513
log()
Definition: remake.cpp:756
input_status next(std::string &)
Definition: remake.cpp:1224
std::string name
Definition: remake.cpp:1171
std::set< std::string > string_set
Definition: remake.cpp:457
static rule_map specific_rules
Definition: remake.cpp:622
static int waiting_jobs
Definition: remake.cpp:671
string_list wdeps
Like deps, except that they are not registered as dependencies.
Definition: remake.cpp:547
Target is up-to-date.
Definition: remake.cpp:510
static time_t now
Definition: remake.cpp:714
ref_ptr(ref_ptr const &p)
Definition: remake.cpp:477
static status_t const & get_status(std::string const &target)
Definition: remake.cpp:1915
std::map< std::string, string_list > variable_map
Definition: remake.cpp:503
Definition: remake.cpp:752
static void load_rule(std::istream &in, std::string const &first)
Definition: remake.cpp:1579
static std::string first_target
Definition: remake.cpp:699
std::list< rule_t > rule_list
Definition: remake.cpp:552
static void server_loop()
Definition: remake.cpp:2729
static bool build_failure
Definition: remake.cpp:687
static client_list clients
Definition: remake.cpp:638
static std::string read_word(std::istream &in, bool detect_equal=true)
Definition: remake.cpp:1098
string_list deps
Dependencies used for an implicit call to remake at the start of the script.
Definition: remake.cpp:546
ref_ptr(T const &t)
Definition: remake.cpp:476
string_list::const_iterator sufi
Definition: remake.cpp:1345
bool append
Definition: remake.cpp:534
static bool skip_eol(std::istream &in, bool multi=false)
Definition: remake.cpp:1023
static bool read_words(input_generator &in, string_list &res)
Definition: remake.cpp:1264
static void register_scripted_rule(rule_t const &rule)
Definition: remake.cpp:1549
string_set deps
Definition: remake.cpp:498
static void init_working_dir()
Definition: remake.cpp:853
static std::ostream & operator<<(std::ostream &out, escape_string const &se)
Definition: remake.cpp:812
static bool keep_going
Definition: remake.cpp:650
static pid_job_map job_pids
Definition: remake.cpp:632
char ** environ
static std::string normalize_abs(std::string const &s, std::string const &p)
Definition: remake.cpp:909
assign_map assigns
Assignment of variables.
Definition: remake.cpp:548
static bool echo_scripts
Definition: remake.cpp:709
string_list::const_iterator vend
Definition: remake.cpp:1172
input_generator(std::istream &i, variable_map const *lv, bool e=false)
Definition: remake.cpp:1216
Definition: remake.cpp:1154
static status_map status
Definition: remake.cpp:612
input_generator gen
Definition: remake.cpp:1343
static std::string working_dir
Definition: remake.cpp:719
static std::string prefix_dir
Definition: remake.cpp:724
T & operator*() const
Definition: remake.cpp:487
bool open
Definition: remake.cpp:754
std::ostream & operator()()
Definition: remake.cpp:759
static void sigint_handler(int)
Definition: remake.cpp:744
std::map< std::string, ref_ptr< dependency_t > > dependency_map
Definition: remake.cpp:501
variable_map vars
Values of local variables.
Definition: remake.cpp:564
static volatile sig_atomic_t got_SIGCHLD
Definition: remake.cpp:737
input_status
Definition: remake.cpp:1150
static int running_jobs
Definition: remake.cpp:663
static void update_status(std::string const &target)
Definition: remake.cpp:1985
variable_generator(std::string const &, variable_map const *)
Definition: remake.cpp:1177
Target is missing or obsolete.
Definition: remake.cpp:511
std::string stem
Pattern used to instantiate the generic rule, if any.
Definition: remake.cpp:563
std::istream & in
Definition: remake.cpp:1212
string_list::const_iterator prei
Definition: remake.cpp:1289
static bool handle_clients()
Definition: remake.cpp:2402
static void accept_client()
Definition: remake.cpp:2595
string_list::const_iterator vcur
Definition: remake.cpp:1172
int socket_t
Definition: remake.cpp:446
bool still_open
Definition: remake.cpp:783
client_t()
Definition: remake.cpp:593
static bool show_targets
Definition: remake.cpp:704
addsuffix_generator(input_generator const &, bool &)
Definition: remake.cpp:1352
static void merge_rule(rule_t &dest, rule_t const &src)
Definition: remake.cpp:1784
Build failed for target.
Definition: remake.cpp:515
log debug
Definition: remake.cpp:779
string_list targets
Files produced by this rule.
Definition: remake.cpp:545
static bool has_free_slots()
Definition: remake.cpp:2379
#define DEBUG
Definition: remake.cpp:793
Target has an obsolete dependency.
Definition: remake.cpp:512
std::string script
Shell script for building the targets.
Definition: remake.cpp:549
content(T const &t)
Definition: remake.cpp:472
Target was successfully rebuilt.
Definition: remake.cpp:514
static bool changed_prefix_dir
Definition: remake.cpp:729
static void skip_spaces(std::istream &in)
Definition: remake.cpp:1002
input_status next(std::string &)
Definition: remake.cpp:1196
#define DEBUG_close
Definition: remake.cpp:795
string_set running
Targets being built.
Definition: remake.cpp:590
~ref_ptr()
Definition: remake.cpp:478
static void normalize_list(string_list &l, std::string const &w, std::string const &p)
Definition: remake.cpp:982
std::map< std::string, ref_ptr< rule_t > > rule_map
Definition: remake.cpp:554
static void skip_empty(std::istream &in)
Definition: remake.cpp:1012
static void find_generic_rule(job_t &job, std::string const &target)
Definition: remake.cpp:1823
socket_t socket
Socket used to reply to the client (invalid for pseudo clients).
Definition: remake.cpp:586
content * ptr
Definition: remake.cpp:474
static void complete_job(int job_id, bool success)
Definition: remake.cpp:2052
ref_ptr()
Definition: remake.cpp:475
static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
Definition: remake.cpp:1807
generator * nested
Definition: remake.cpp:1213
static socket_t socket_fd
Definition: remake.cpp:682
rule_t rule
Original rule.
Definition: remake.cpp:562
string_list value
Definition: remake.cpp:535
static int expect_token(std::istream &in, int mask)
Definition: remake.cpp:1052
input_generator gen
Definition: remake.cpp:1287
string_list pending
Targets not yet started.
Definition: remake.cpp:589
bool active
Definition: remake.cpp:754
virtual ~generator()
Definition: remake.cpp:1162
static void usage(int exit_status)
Definition: remake.cpp:2930
static void client_mode(char *socket_name, string_list const &targets)
Definition: remake.cpp:2838
static int job_counter
Definition: remake.cpp:677
static void finalize_job(pid_t pid, bool res)
Definition: remake.cpp:2714
std::list< client_t > client_list
Definition: remake.cpp:596
std::map< pid_t, int > pid_job_map
Definition: remake.cpp:569
std::map< std::string, assign_t > assign_map
Definition: remake.cpp:538
bool failed
Whether some targets failed in mode -k.
Definition: remake.cpp:588
static status_e start(std::string const &target, client_list::iterator &current)
Definition: remake.cpp:2291
static void load_dependencies(std::istream &in)
Definition: remake.cpp:1415
static void load_rules(std::string const &remakefile)
Definition: remake.cpp:1713
static void register_transparent_rule(rule_t const &rule, string_list const &targets)
Definition: remake.cpp:1506
time_t last
Last-modified date.
Definition: remake.cpp:524
input_status next(std::string &)
Definition: remake.cpp:1306
status_e status
Actual status.
Definition: remake.cpp:523
static rule_list generic_rules
Definition: remake.cpp:617
int main(int argc, char *argv[])
Definition: remake.cpp:2956
virtual input_status next(std::string &)=0
static dependency_map dependencies
Definition: remake.cpp:607
std::string suf
Definition: remake.cpp:1291
static std::string prepare_script(job_t const &job)
Definition: remake.cpp:2087
static void sigchld_handler(int)
Definition: remake.cpp:739
variable_map const * local_variables
Definition: remake.cpp:1214
bool earliest_exit
Definition: remake.cpp:1215
string_list pre
Definition: remake.cpp:1288
string_list suf
Definition: remake.cpp:1344